Papers with Robustness in NLP
Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systems (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to enable skill routing do not scale in terms of the number of skills and skill on-boarding. |
| Approach: | They propose a model-based approach to enable natural conversation by allowing frequent policy updates . they propose an annotation-based system, rule-based model, and bandit-based learning . |
| Outcome: | The proposed method is scalable and cost-effective, the authors show . they show that it can improve the user experience without abrupt policy changes . |
Detecting Subtle Sense Shift with Polysemy-Aware Trends (2026.eacl-short)
Copied to clipboard
| Challenge: | Existing research on lexical semantic change focuses on century-scale, well-curated corpora and binary "changed / unchanged" judgements. |
| Approach: | They propose a language-independent pipeline that detects word-sense shifts in large, time-stamped web corpora. |
| Outcome: | The proposed pipeline detects word-sense shifts in large, time-stamped web corpora. |
Security Challenges in Natural Language Processing Models (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | Large-scale natural language processing models are vulnerable to security issues, such as backdoor attacks, private data leakage, and imitation attacks. |
| Approach: | They will dive into three emerging security issues in NLP research, i.e., backdoor attacks, private data leakage, and imitation attacks. |
| Outcome: | This tutorial will cover three emerging security issues in NLP research, i.e., backdoor attacks, private data leakage, and imitation attacks. |
RAVEN: Robust Advertisement Video Violation Temporal Grounding via Reinforcement Reasoning (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for detecting ads video violations lack precise temporal grounding, noisy annotations, and limited generalization. |
| Approach: | They propose a framework that integrates curriculum reinforcement learning with large language models to enhance reasoning and cognitive capabilities for violation detection. |
| Outcome: | The proposed framework achieves superior performance in violation category accuracy and temporal interval localization. |
WorkForceAgent-R1: Incentivizing Reasoning Capability in LLM-based Web Agents via Reinforcement Learning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing web agents relying on supervised fine-tuning struggle with generalization and robustness due to insufficient reasoning capabilities when handling the inherently dynamic nature of web interactions. |
| Approach: | They propose a large language model-empowered web agent that trains using a rule-based reinforcement learning framework to enhance single-step reasoning and planning for business-oriented web navigation tasks. |
| Outcome: | The proposed agent outperforms baseline LLM-based agents on the WorkArena benchmark by 10.26–16.59%. |
Input-specific Attention Subnetworks for Adversarial Detection (2022.findings-acl)
Copied to clipboard
| Challenge: | a new method to prune attention heads is proposed for adversarial detection . attention heads in models such as BERT are over-provisioned and can be pruned . |
| Approach: | They propose a method to construct input-specific attention subnetworks from which three features are extracted to discriminate between authentic and adversarial inputs. |
| Outcome: | The proposed method significantly improves state-of-the-art adversarial detection accuracy on 10 NLU datasets with 11 different adversarials. |
Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Models (2026.acl-long)
Copied to clipboard
Yifu Chen, Shengpeng Ji, Zhengqing Liu, Qian Chen, Wen Wang, Ziqing Wang, Yangzhuo Li, Tianle Liang, Zhou Zhao
| Challenge: | Reinforcement learning (RL) has improved text- and vision-language models, but its application in SDMs is hindered. |
| Approach: | They propose a dual-axis Generative Reward Model that provides semantic quality and interaction timing for SDMs. |
| Outcome: | The proposed model achieves state-of-the-art performance on interaction-quality assessment across a wide spectrum of datasets. |
Span-Selective Linear Attention Transformers for Effective and Robust Schema-Guided Dialogue State Tracking (2023.acl-long)
Copied to clipboard
| Challenge: | Existing schema-guided dialogue state tracking models do not account for schema variations and are not generalized to unseen services. |
| Approach: | They propose a new architecture which allows for rich attention among descriptions and history while keeping computation costs constrained. |
| Outcome: | The proposed model outperforms the more than 30x larger D3ST-XXL model on the SGD-X benchmark by 5.0 points. |
Robustness Gym: Unifying the NLP Evaluation Landscape (2021.naacl-demos)
Copied to clipboard
| Challenge: | Existing tools cater to specialized set of evaluations and provide no clear way to leverage or share findings from prior evaluations. |
| Approach: | They propose a toolkit that unifies 4 evaluation paradigms to provide a common platform for evaluation. |
| Outcome: | The proposed evaluation toolkit unifies 4 evaluation paradigms and is under active development. |
Guardrails and Security for LLMs: Safe, Secure and Controllable Steering of LLM Applications (2025.acl-tutorials)
Copied to clipboard
Traian Rebedea, Leon Derczynski, Shaona Ghosh, Makesh Narsimhan Sreedhar, Faeze Brahman, Liwei Jiang, Bo Li, Yulia Tsvetkov, Christopher Parisien, Yejin Choi
| Challenge: | Pretrained generative models provide novel ways for users to interact with computers. |
| Approach: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs along with evaluation methodologies and a detailed security assessment protocol. |
| Outcome: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs, along with evaluation methodologies and a detailed security assessment protocol. |
Generating Text through Adversarial Training Using Skip-Thought Vectors (N19-3)
Copied to clipboard
| Challenge: | Existing approaches to use word embeddings for text generation have been limited. |
| Approach: | They propose to use GANs with word embeddings to reproduce writing style in text . they use a sentence embeddable vector to model people's way of expression . |
| Outcome: | The proposed model outperforms baseline text generation networks across several metrics including BLEU-n, METEOR and ROUGE. |
Quality-Aware Adversarial Ensemble for Singer Identification in 1960s Tamil Film Music (2026.eacl-srw)
Copied to clipboard
| Challenge: | 1960s Tamil film music lacks adequate metadata identifying playback singers in archival recordings. |
| Approach: | They propose a quality-aware adversarial ensemble approach based on variable audio degradation and instrumentation leakage confounding singer-specific features. |
| Outcome: | The proposed approach achieves 96.2% accuracy and 2.0% EER on a held-out test set of 52 clips. |
Layerwise universal adversarial attack on NLP models (2023.findings-acl)
Copied to clipboard
| Challenge: | a new technique for layerwise UATs searches hidden layers of a network for universal adversarial triggers . a previous study showed that adversarials can fool models by perturbing samples that leave the ground truth label unchanged but can modify model prediction drastically. |
| Approach: | They propose a new approach to construct layerwise UATs by perturbing hidden layers of a network and propose LUATs that are more efficient than vanilla UAT methods. |
| Outcome: | The proposed method provides better transferability in a model-to-model setting with an average gain of 9.3% in fooling rate over baseline. |
Commentator: A Code-mixed Multilingual Text Annotation Framework (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Existing annotation tools fail to address multilingual datasets efficiently. |
| Approach: | They introduce a code-mixed multilingual text annotation framework, COMMENTATOR . they perform robust qualitative human-based evaluations to showcase its effectiveness . |
| Outcome: | The proposed framework performs faster than baseline annotations in Hinglish and Hindi. |
Enhancing Adversarial Transferability in Visual-Language Pre-training Models via Local Shuffle and Sample-based Attack (2025.findings-naacl)
Copied to clipboard
| Challenge: | Visual-Language Pre-training (VLP) models are vulnerable to adversarial examples . previous studies have focused on improving adversariality of models . |
| Approach: | They propose a local shuffle and sample-based attack that randomly shufts one of the local image blocks and generates adversarial images and samples around them. |
| Outcome: | The proposed attack outperforms other advanced attacks on Large Vision-Language Models and outperformed previous attacks on Visual-Langue Pre-training models. |
Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Existing evaluation tools for general Visual Question Answering (VQA) systems are limited to answering accuracy, but they can be used to evaluate performance in real-world scenarios. |
| Approach: | They propose a browser-based benchmarking tool with an API for easy integration of new models and datasets to keep up with the fast-changing landscape of VQA. |
| Outcome: | The proposed tool tests generalization capabilities of models across multiple datasets and includes metrics that measure biases and uncertainty to further explain model behavior. |
Analyzing Dynamic Adversarial Training Data in the Limit (2022.findings-acl)
Copied to clipboard
| Challenge: | Dynamic adversarial data collection (DADC) can be used to build models that are robust across a wide range of test inputs. |
| Approach: | They propose to run Dynamic adversarial data collection over many rounds to maximize its training-time benefits. |
| Outcome: | The proposed model makes 26% fewer errors on the premise paragraphs compared to models trained on non-adversarial examples. |
Towards Reliable and Practical Phishing Detection (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing datasets lack size and diversity, with only 609 voice phishing samples available in Korean and 638 smishing instances available in English. |
| Approach: | They propose to use a Korean dataset to construct a reliable phishing detection system using language models to evaluate the model's in-domain and unseen attack detection performance. |
| Outcome: | The proposed system performs reasonably well in voice and unseen attacks while smishing detection remains challenging. |
Reliable Use of Lemmas via Eligibility Reasoning and Section-Aware Reinforcement Learning (2026.acl-short)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) perform strongly on mathematical benchmarks but often import conclusions without validating assumptions. |
| Approach: | They propose a model that encodes a lemma specification and trains with reinforcement learning and section-aware loss masking to assign penalty to the section responsible for errors. |
| Outcome: | The proposed model performs well on benchmarks but often misapplyes lemmas . the model is able to encode the specification and train with reinforcement learning . |
A Robustly Optimized BMRC for Aspect Sentiment Triplet Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Aspect sentiment triplet extraction (ASTE) is a challenging subtask in aspect-based sentiment analysis. |
| Approach: | They propose a bidirectional machine reading comprehension method to extract triplets of aspects, opinions and sentiments with complex correspondence from the context. |
| Outcome: | The proposed method achieves state-of-the-art on multiple benchmark datasets. |
Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to Automated Essay Scoring (AES) are not well-suited to capture adversarially crafted input of grammatical but incoherent sequences of sentences. |
| Approach: | They propose a neural model of local coherence that can effectively learn connectedness features between sentences. |
| Outcome: | The proposed approach strengthens the validity of neural essay scoring models. |
Computational Ad Hominem Detection (P19-2)
Copied to clipboard
| Challenge: | ad hominem attacks are introduced in debates as an easy win, but their impact on argumentation is limited . a machine learning approach to detect the personal attack is insufficient, we show . |
| Approach: | They propose a machine learning approach that detects ad hominem attacks using social media data . they propose TF-IDF approaches that are insufficient to detect the personal attack . |
| Outcome: | The proposed method has a recall of 80% for a social media data source. |
Enhance Robustness of Sequence Labelling with Masked Adversarial Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Adversarial training (AT) has shown strong regularization effects on deep learning algorithms by introducing small input perturbations to improve model robustness. |
| Approach: | They propose to use adversarial training to improve robustness from contextual information in sequence labelling tasks by masking or replacing some words in the sentence. |
| Outcome: | The proposed method shows significant improvements on accuracy and robustness of sequence labelling on CoNLL 2000 and 2003 benchmarks. |
Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety (2026.acl-long)
Copied to clipboard
Can Jin, Rui Wu, Tong Che, Qixin Zhang, Hongwu Peng, Jiahui Zhao, Zhenting Wang, Wenqi Wei, Ligong Han, Zhao Zhang, Yuan Cao, Ruixiang Tang, Dimitris N. Metaxas
| Challenge: | OpenAI introduces deliberative alignment (DA) to enhance safety of its o-series models, but effectiveness of this approach in open-source LLMs is understudied. |
| Approach: | They propose a case-augmented deliberative alignment method for large language models . they propose to use reinforcement learning on self-generated safety reasoning chains . |
| Outcome: | The proposed method avoids narrowly enumerated rules and allows broader adaptability. |
Toward Stronger Textual Attack Detectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | despite the high performance of deep learning techniques, trained models remain vulnerable to adversarial attacks . authors present LAROUSSE, STAKEOUT and other approaches to detect adversarials . LARousSE is unsupervised, hyperparameter free and non-differentiable . |
| Approach: | They propose a framework to detect adversarial attacks and an extended benchmark to test them . they demonstrate that LAROUSSE outperforms previous methods and allows to identify interesting factor of detection rate variations. |
| Outcome: | The proposed framework outperforms existing methods and allows to identify interesting factor of detection rate variations. |
Bridging Distribution Gap via Semantic Rewriting with LLMs to Enhance OOD Robustness (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning on indistribution data fail to provide robustness against distribution shifts limiting the practical deployment of LLMs in dynamic real-world scenarios. |
| Approach: | They propose a method that leverages the flexibility of LLMs to align both in-distribution (ID) and OOD data with the LLM's distributions. |
| Outcome: | The proposed method outperforms fine-tuning methods on OOD tasks and benchmark datasets. |
Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Traditional adversarial examples involve finding a small perturbation that does not change an input’s true label but confuses the classifier into outputting a different prediction. |
| Approach: | They propose to use contrastive learning to increase model robustness against fickle adversarial examples by reducing the vulnerability of adversarials to fickle ones. |
| Outcome: | The proposed method improves model robustness against fickle and obstinate adversarial examples. |
Two Heads are Better than One: Nested PoE for Robust Defense Against Multi-Backdoors (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing defense mechanisms assume that only one type of trigger is adopted by the attacker, while defending against multiple simultaneous and independent trigger types necessitates general defense frameworks. |
| Approach: | They propose a framework that uses a mixture of experts as a trigger-only ensemble to defend against multiple trigger types. |
| Outcome: | The proposed framework defends against multiple trigger types in a single ensemble and in combination of models. |
Adversarial Scrubbing of Demographic Information for Text Classification (2021.emnlp-main)
Copied to clipboard
Somnath Basu Roy Chowdhury, Sayan Ghosh, Yiyuan Li, Junier Oliva, Shashank Srivastava, Snigdha Chaturvedi
| Challenge: | Existing frameworks to debias contextual representations can encode undesirable attributes, like demographic associations of the users, while being trained for an unrelated task. |
| Approach: | They propose an adversarial learning framework to debias contextual representations by encoding undesirable attributes while being trained for an unrelated task. |
| Outcome: | The proposed framework debiases representations on 8 datasets while remaining informative on the target task. |
IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages (2026.eacl-industry)
Copied to clipboard
| Challenge: | Indic Jailbreak Robustness (IJR) is a judge-free benchmark for adversarial safety across 12 languages. |
| Approach: | They propose a judge-free benchmark for adversarial safety across 12 languages . they find contracts inflate refusals but do not stop jailbreaks . |
| Outcome: | The proposed benchmarks cover 45,216 prompts in JSON and Free tracks. |
Robust Learning for Multi-party Addressee Recognition with Discrete Addressee Codebook (2023.acl-short)
Copied to clipboard
| Challenge: | Existing addressee recognition models suffer from the issue of robustness when applied in real-world scenes. |
| Approach: | They propose a method which discretes addressees into a character codebook and makes it robust in a noisy environment. |
| Outcome: | The proposed method represents open set addressees and is robust even in noisy environments. |
Towards Building a Robust Toxicity Predictor (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent studies have focused on robustness of toxicity language predictors, but this is problematic for real-world toxicity detection. |
| Approach: | They propose a novel adversarial attack that exploits greedy search strategies to fool toxic text classifiers. |
| Outcome: | The proposed attack can detect weaker toxicity language detectors even against unseen attacks. |
An Empirical Study on Adversarial Attack on NMT: Languages and Positions Matter (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to generating NMT adversarial examples inject perturbations into source sentences or target translations to improve the robustness of NMT models. |
| Approach: | They investigate adversarial attack on NMT from two aspects: languages (the source vs. the target language) and positions (front v. rear). |
| Outcome: | The proposed approach is more effective than adversarial attacks by sampling positions randomly or according to gradients. |
Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP models are often trained over large untrustworthy datasets, raising the potential for a malicious adversary to compromise model behaviour. |
| Approach: | They propose to mitigate spurious correlations between textual triggers and classification labels by combining them with insertion-based attacks. |
| Outcome: | The proposed defence significantly reduces attack success rates across backdoor attacks and provides a near-perfect defence against insertion-based attacks. |
Attacker’s Noise Can Manipulate Your Audio-based LLM in the Real World (2026.eacl-long)
Copied to clipboard
| Challenge: | a recent study examines the real-world vulnerabilities of audio-based large language models such as Qwen2 . despite their impressive capabilities, large language model's are susceptible to various security exploits . |
| Approach: | They investigate the real-world vulnerabilities of audio-based large language models . they show that an adversary can craft stealthy audio perturbations to manipulate ALLMs . |
| Outcome: | The proposed attacks can be adapted to real-world scenarios and impact innocent users . the proposed attacks are scalable and can be exploited in real-time . |
Improving Arabic Diacritization with Regularized Decoding and Adversarial Training (2021.acl-short)
Copied to clipboard
| Challenge: | Existing studies regard auto-generated knowledge instances as gold references, which limits their effectiveness since they are not always accurate and inferior instances can lead to incorrect predictions. |
| Approach: | They propose to use regularized decoding and adversarial training to appropriately learn from noisy knowledge instances for Arabic diacritization. |
| Outcome: | The proposed model outperforms existing models on two benchmark datasets even with flawed auto-generated knowledge. |
SHARP: Search-Based Adversarial Attack for Structured Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | SHARP is a new attack method for structured prediction models that solves several challenges. |
| Approach: | They propose a black-box adversarial attack method that uses a search-based optimization problem to attack adversarials. |
| Outcome: | The proposed method performs more potent attack than pioneer arts on two structured prediction tasks. |
Out-of-domain Detection based on Generative Adversarial Network (D18-1)
Copied to clipboard
| Challenge: | Existing methods for out-of-domain (OOD) detection require huge effort to collect OOD sentences. |
| Approach: | They propose to use only in-domain (IND) sentences to build a generative adversarial network (GAN) of which the discriminator generates low scores for OOD sentences. |
| Outcome: | The proposed method is most accurate compared to existing methods on multi-domain dialog systems. |
Unveiling the Achilles’ Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have highlighted various neural metrics that align well with human evaluations. |
| Approach: | They propose a black-box adversarial framework that generates strong disagreements between human and victim evaluators. |
| Outcome: | The proposed framework can significantly improve the performance of human and victim evaluators. |
Towards Improving Adversarial Training of NLP Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods for generating NLP adversarial examples involve combinatorial search and expensive sentence encoders for constraining the generated instances. |
| Approach: | They propose to use vanilla adversarial training to train NLP models using a word substitution attack optimized for vanilla adversary training. |
| Outcome: | The proposed approach improves model performance and standard accuracy and can defend against other types of word substitution attacks. |
Common Phone: A Multilingual Dataset for Robust Acoustic Modelling (2022.lrec-1)
Copied to clipboard
| Challenge: | Current state-of-the-art acoustic models can easily comprise more than 100 million parameters. |
| Approach: | They propose to train a gender-balanced, multilingual corpus from 76.000 contributors via Mozilla’s Common Voice project to perform phonetic symbol recognition and validate the quality of the generated phonetic annotation. |
| Outcome: | The proposed model can perform phonetic symbol recognition and validate the quality of the generated phonetic annotation. |
Certified Robustness to Programmable Transformations in LSTMs (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing techniques for certifying robustness of LSTMs and extensions of lsts are prone to adversarial examples. |
| Approach: | They propose an approach to certify robustness of LSTMs and extensions of lstms . they show that their approach can train models more robust to combinations of string transformations - a key advantage of existing certification approaches . |
| Outcome: | The proposed approach can show high certification accuracy of the resulting models. |
Where to Attack: A Dynamic Locator Model for Backdoor Attack in Text Classifications (2022.coling-1)
Copied to clipboard
| Challenge: | BackDoor Attack (BDA) study aims to train a poisoned model with clean data and some trigger-embedded instances to perform normally on normal inputs. |
| Approach: | They propose to train a poisoned model with clean and poisonest inputs . they propose to use triggers to predict those poisonets as target labels . |
| Outcome: | The proposed model can predict P2P dynamically without human intervention. |
Learning Bias-reduced Word Embeddings Using Dictionary Definitions (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing word embeddings have undesirable gender, racial, and religious biases . DD-GloVe is a train-time debiasing algorithm that uses dictionary definitions based on word definitions. |
| Approach: | They propose a dictionary-guided loss function that encourages word embeddings to be similar to their relatively neutral dictionary definition representations. |
| Outcome: | The proposed algorithm can learn word embeddings by leveraging dictionary definitions. |
Improving the Robustness of Summarization Models by Detecting and Removing Input Noise (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive summarization models are typically evaluated using test data that is identically distributed as training data. |
| Approach: | They propose a method to detect and remove input noise from documents to be summarized without extra training or auxiliary models. |
| Outcome: | The proposed method recovers a large fraction of the loss in performance, sometimes as large as 11 ROUGE-1 points, without extra training, auxiliary models, or prior knowledge of the type of noise. |
Robust Question Answering Through Sub-part Alignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Current textual question answering models fail to generalize to out-of-domain settings. |
| Approach: | They propose to decompose question and context into smaller units and align them to find the answer. |
| Outcome: | The proposed model is more robust than the standard BERT QA model on adversarial and out-of-domain datasets. |
Enhancing Neural Models with Vulnerability via Adversarial Attack (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on adversarial attack to improve performance of NLSM tasks has not been done. |
| Approach: | They propose a general two-stage training framework to enhance neural models with Vulnerability via adversarial attack. |
| Outcome: | The proposed framework improves neural models with Vulnerability via adversarial attack on NLSM datasets. |
SMART: Scalable Multilingual Approach for a Robust TOD System (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing TOD frameworks face significant challenges in handling unstructured information, providing multilingual support, and engaging proactively. |
| Approach: | They propose a novel TOD framework that combines traditional pipeline elements with modern agent-based approaches and features a simplified dialogue state, intelligent clarification mechanisms, and a unified natural language generation component that eliminates response redundancy. |
| Outcome: | The proposed framework outperforms baseline systems across key metrics and integrates in an e-commerce store. |
Adversarial Decoding: Generating Readable Documents for Adversarial Objectives (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for generating adversarial documents produce gibberish that is easy to detect and filter out. |
| Approach: | They propose a generic text generation technique that produces readable adversarial documents . they demonstrate that adversarials can be used for different objectives . |
| Outcome: | The proposed technique outperforms existing methods while producing readable documents for adversarial objectives. |
Reasoning or Knowledge: Stratified Evaluation of Biomedical LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Medical reasoning in large language models is a complex cognitive process through which clinicians interpret patient data and make diagnostic and therapeutic decisions. |
| Approach: | They propose an evaluation framework that disentangles knowledge recall from reasoning by training a PubMedBERT-based classifier and applying it to 11 widely used biomedical QA benchmarks. |
| Outcome: | The proposed evaluation framework disentangles knowledge recall from reasoning by training a PubMedBERT-based classifier and applying it to 11 widely used biomedical QA benchmarks. |
Achieving Model Robustness through Discrete Adversarial Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Discrete adversarial attacks are symbolic perturbations to a language input that preserve the output label but lead to predicting error. |
| Approach: | They propose a discrete adversarial attack based on best-first search and random sampling attacks that are not based upon expensive search procedures. |
| Outcome: | The proposed attack outperforms offline augmentation and speedups on three datasets. |
Evaluating the Robustness of Neural Language Models to Input Perturbations (2021.emnlp-main)
Copied to clipboard
| Challenge: | High-performance neural language models have achieved state-of-the-art results on a wide range of NLP tasks, but results for common benchmark datasets often do not reflect model reliability and robustness when applied to noisy, real-world data. |
| Approach: | They propose to implement character-level and word-level perturbation methods to simulate scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained. |
| Outcome: | The proposed methods simulate scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained. |
Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning (2021.acl-short)
Copied to clipboard
| Challenge: | In news articles the lead bias dominates the learning signals for neural extractive summarizations, severely limiting their performance on data with different or even no bias. |
| Approach: | They propose a method to demote the lead bias in news and make the model focus more on the content semantics. |
| Outcome: | The proposed method can demote the model’s learned lead bias and improve its generality on out-of-distribution data with little to no performance loss on in-difference data. |
LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for post-training quantization (PTQ) are limited by the complexity of the quantization parameter and performance degradations when tested on unseen datasets. |
| Approach: | They propose a learnable smooth-based PTQ framework that allows for rapid adaptation during testing. |
| Outcome: | The proposed framework improves performance on unseen datasets and reduces memory constraints. |
Extreme Miscalibration and the Illusion of Adversarial Robustness (2024.acl-long)
Copied to clipboard
| Challenge: | emergence of the Adversarial Training paradigm has shown some success in training models to be more robust to these small adversarial perturbations. |
| Approach: | They propose to use adversarial examples to detect adversarials by miscalibrating models to mask gradients in a way that interferes with adversarial attack search methods. |
| Outcome: | The proposed model gains are an illusion of robustness (IOR) and urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations to ensure that any observed gains are genuine. |
Distributionally Robust Finetuning BERT for Covariate Drift in Spoken Language Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Covariate drift can occur when there is a drift between training and testing regarding what users request or how they request it. |
| Approach: | They propose a method that exploits natural variations in data to create a covariate drift in spoken language understanding datasets. |
| Outcome: | The proposed method improves robustness against covariate drift in spoken language understanding (SLU) it shows that a state-of-the-art model suffers performance loss under this drift. |
On the Robustness of Question Rewriting Systems to Questions of Varying Hardness (2022.acl-long)
Copied to clipboard
| Challenge: | entailment : absence of questions classified based on their rewriting hardness or difficulty . enactment of QR system to rewrite context-dependent questions in CQA requires context knowledge . |
| Approach: | They propose a heuristic method to automatically classify questions into subsets of varying hardness . they then conduct a human evaluation to annotate the rewriting hardness of questions . |
| Outcome: | The proposed learning framework improves the overall performance compared to baselines. |
Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing gradient-based attacks quantize all tokens in a text at once, which creates a significant gap between adversarial loss for continuous and discrete text representations. |
| Approach: | They propose a gradient-based attack that quantizes tokens one by one and reoptimizes adversarial example after each quantization. |
| Outcome: | The proposed method outperforms other approaches on various natural language processing tasks. |
Reducing Spurious Correlations for Answer Selection by Feature Decorrelation and Language Debiasing (2022.coling-1)
Copied to clipboard
| Challenge: | Existing deep neural models rely on spurious correlations between prediction labels and input features, which in general suffer from robustness and generalization. |
| Approach: | They propose a feature decorrelation module to remove feature dependencies and reduce spurious correlations by learning a weight for each instance at the training phase. |
| Outcome: | The proposed method improves the robustness of the neural ANswer selection models from the sample and feature perspectives. |
Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognition (2022.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) systems perform well on in-distribution data, but perform poorly on examples drawn from a shifted distribution. |
| Approach: | They propose to use expert-guided heuristics to change entity tokens and their contexts to alter their entity types as adversarial attacks. |
| Outcome: | The proposed model significantly improves performance on the challenging set and out-of-domain generalization. |
Iron Sharpens Iron: Defending Against Attacks in Machine-Generated Text Detection with Adversarial Training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing MGT detectors are vulnerable to simple perturbations and adversarial attacks. |
| Approach: | They propose an adversarial framework for training a robust machine-generated text detector called GREedy Adversary PromoTed DefendER. |
| Outcome: | The proposed framework reduces the Attack Success Rate (ASR) by 0.67% compared with SOTA defense methods. |
DemonAgent: Dynamically Encrypted Multi-Backdoor Implantation Attack on LLM-based Agent (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new method for detecting advanced backdoors is proposed to bypass safety audits. |
| Approach: | They propose a backdoor implantation strategy that introduces dynamic encryption to bypass safety audits. |
| Outcome: | The proposed method achieves an attack success rate approaching 100% while maintaining a detection rate of 0%. |
Counter-Argument Generation by Attacking Weak Premises (2021.findings-acl)
Copied to clipboard
| Challenge: | a recent work explores the generation of counter-arguments by undermining one of its premises . identifying the argument's weak premises is key to effective countering, we hypothesize . |
| Approach: | They propose a pipeline approach that first assesses the argument's weak premises and generates a counter-argument undermining the weakest among them. |
| Outcome: | The proposed approach undermins arguments by attacking weak premises . human annotators favor the proposed approach over state-of-the-art approaches . |
Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on this topic focus on the robustness of specific detectors or particular attack methods. |
| Approach: | They stress test the detectors’ robustness to malicious attacks under realistic scenarios using LLMs and metric-based detectors. |
| Outcome: | The proposed methods are based on a set of LLM-based models and their performance is compared under different budget levels. |
TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System (2025.findings-naacl)
Copied to clipboard
Zeyu Zhang, Jianxun Lian, Chen Ma, Yaning Qu, Ye Luo, Lei Wang, Rui Li, Xu Chen, Yankai Lin, Le Wu, Xing Xie, Ji-Rong Wen
| Challenge: | Trending topics bring in a new channel for poisoning attacks, resulting in negative impacts on society. |
| Approach: | They propose an LLM-based multi-agent system to simulate trending topics in social media . they propose a time-aware interaction mechanism, centralized message dissemination, and an interactive system . |
| Outcome: | The proposed system simulates trending topics under poisoning attacks on social media platforms. |
Typos that Broke the RAG’s Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on the robustness of Large Language Models (LLMs) overlook the interconnected relationships between RAG components or the potential threats prevalent in real-world databases, such as minor textual errors. |
| Approach: | They propose a novel attack method that exploits vulnerabilities in RAG components and tests its robustness against noisy documents. |
| Outcome: | The proposed method devastates the performance of each component and their synergy, and significantly devases the performance. |
End-to-End Self-Debiasing Framework for Robust NLU Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models incorporate dataset biases leading to strong performance on in-distribution test sets but poor performance on out-of-distortion (OOD) tests. |
| Approach: | They propose a debiasing framework where the shallow representations of the main model are used to derive a bias model and both models are trained simultaneously. |
| Outcome: | The proposed framework outperforms existing approaches on three well-studied NLU tasks while still delivering high in-distribution performance. |
Exploring How Generative Adversarial Networks Learn Phonological Representations (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies in natural language processing (NLP) have demonstrated two generic trends: neural networks dominate language-specific machine learning models; the interpretability of these models is limited that the language representation they learned might not align to human language. |
| Approach: | They propose to use a phonological feature-learning architecture to encode contrastive and non-contrastive nasality in French and English vowels. |
| Outcome: | The proposed architecture encodes contrastive and non-contrastive nasality in French and English vowels. |
An Evaluation Dataset and Strategy for Building Robust Multi-turn Response Selection Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multi-turn response selection models have shown comparable performance to humans in several benchmark datasets, but in the real environment, they often have weaknesses, such as giving the highest score to the wrong response candidate containing several keywords related to the context. |
| Approach: | They propose to build a robust multi-turn response selection model in an adversarial environment and to use it to evaluate weaknesses. |
| Outcome: | The proposed model makes incorrect predictions based heavily on superficial patterns without a comprehensive understanding of the context. |
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)
Copied to clipboard
Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong
| Challenge: | Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise. |
| Approach: | They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble. |
| Outcome: | The proposed technique improves ensemble performance and robustness against erroneous signals. |
Systematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing Perspective (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on robustness-like metamorphic relations, which limit the scope of linguistic properties they can test. |
| Approach: | They propose three new classes of metamorphic relations which address the properties of systematicity, compositionality and transitivity. |
| Outcome: | The proposed methods show that metamorphic models do not always behave according to expected linguistic properties. |
Backdoor NLP Models via AI-Generated Text (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing attacks disregard fluency and semantic fidelity of poisoned text, rendering it easily detectable. |
| Approach: | They propose to use AI-generated poisoned text to attack NLP models by establishing covert associations between trigger patterns and target labels without affecting normal accuracy. |
| Outcome: | The proposed method achieves effective attacks while maintaining fluency and semantic similarity across all scenarios. |
DocAssistant: Integrating Key-region Reading and Step-wise Reasoning for Robust Document Visual Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing document understanding models focus on key information and generate answers straightforwardly . existing models ignore evidence from source documents and lack interpretability . |
| Approach: | They propose a visual encoder that fuses text into visual encoded visual encodes . they use multimodal large language models as data generators and checkers to generate step-wise question-and-answer pairs for document images. |
| Outcome: | The proposed model can answer step-wise questions without compromising the performance of the original model. |
Adversarial Learning of Poisson Factorisation Model for Gauging Brand Sentiment in User Reviews (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models for sentiment-topic extraction assume topics are grouped under discrete sentiment categories such as ‘positive’, ‘negative’ and ‘neural’. |
| Approach: | They propose a Brand-Topic Model which aims to detect brand-associated polarity-bearing topics from product reviews. |
| Outcome: | The proposed model outperforms existing models on Amazon reviews and shows that it is more coherent and unique than existing models. |
DINER: Debiasing Aspect-based Sentiment Analysis with Multi-variable Causal Inference (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to debiase ABSA focus on single-variable causal inference . aspect-based sentiment analysis models are prone to learn spurious correlations from annotation biases . |
| Approach: | They propose a framework based on multivariable causal inference for debiasing ABSA . they propose to model different types of biases based upon different causal intervention methods . |
| Outcome: | The proposed framework tackles different types of biases based on different intervention methods. |
Adversarial Textual Robustness on Visual Dialog (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study evaluated the robustness of visual dialog models against textual attacks. |
| Approach: | They aim to understand how multimodal input components contribute to robustness . they also evaluate how to generate adversarial test examples which fool the model . |
| Outcome: | The proposed model is more robust when it encodes dialog history than when it does not. |
Universal Adversarial Triggers for Attacking and Analyzing NLP (D19-1)
Copied to clipboard
| Challenge: | Using adversarial triggers, a model can produce a specific prediction . adversarial attacks are useful for evaluation and interpretation . |
| Approach: | They propose a gradient-guided search over tokens that finds short adversarial triggers that successfully trigger the target prediction. |
| Outcome: | The proposed algorithm finds short trigger sequences that successfully trigger the target prediction. |
Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models require additional annotations to adjust to shifted distributions. |
| Approach: | They propose an algorithm that allows LLMs and reward models to update alternatively via a min-max game to improve their alignment. |
| Outcome: | The proposed framework improves existing alignment baselines in terms of LLM helpfulness and harmlessness. |
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated increasing power, but they also have vulnerabilities. |
| Approach: | They propose a black-box attack that targets the cognitive structure and processes of large language models (LLMs) they propose defending cognitive overload attacks from three perspectives. |
| Outcome: | The proposed attack is a black-box attack with no need for knowledge of model architecture or access to model weights. |
Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction are data-hungry and it is hard to train a seq2seq model with good performance without suf-Clean. |
| Approach: | They propose a method inspired by adversarial training to generate more meaningful and valuable training examples by continually identifying weak spots of a model and to enhance the model by gradually adding adversarials to the training set. |
| Outcome: | The proposed method improves generalization and robustness of GEC models by adding adversarial examples to the training set. |
How Does Context Matter? On the Robustness of Event Detection with Context-Selective Mask Generalization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on improving the overall performance of an ED model, but few consider the robustness of an existing model. |
| Approach: | They propose a new training mechanism that can effectively mine context-specific patterns for learning and robustify an ED model. |
| Outcome: | The proposed model can learn a complementary predictive bias with most ED models that use full context for feature learning. |
Adversarial Examples for Evaluating Math Word Problem Solvers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing MWP solvers do not understand language and its relation with numbers, and their accuracy is unclear. |
| Approach: | They propose two methods to generate adversarial attacks to evaluate the robustness of existing MWP solvers. |
| Outcome: | The proposed method reduces the accuracy of existing MWP solvers by over 40% on two benchmark datasets. |
Machine Translation Robustness to Natural Asemantic Variation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing machine translation models struggle with noisy data and tail-end words and phrases. |
| Approach: | They introduce and formalize a class of noise and variation that preserves meaning in the target language. |
| Outcome: | The proposed model can perform better on natural asemantic variation (NAV) the proposed model is robust to a variety of perturbations, but not all of them are achieved with organic variations. |
Learning Invariant Representation Improves Robustness for MRC Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to improve machine reading comprehension models are vulnerable and not robust to adversarial examples. |
| Approach: | They propose to construct positive example pairs which have same answer by augmentation and then introduce stability and contrastive loss to improve invariance of representation. |
| Outcome: | The proposed approach boosts the robustness of QA models across different tasks and attack sets significantly and consistently. |
Improving Robustness of Language Models from a Geometry-aware Perspective (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have found that removing the norm-bounded projection and increasing search steps in adversarial training can significantly improve robustness. |
| Approach: | They propose friendly adversarial data augmentation and geometry-aware adversarial training to achieve stronger robustness using fewer search steps. |
| Outcome: | The proposed method can obtain stronger robustness using fewer steps than existing methods. |
BitAbuse: A Dataset of Visually Perturbed Texts for Defending Phishing Attacks (2025.findings-naacl)
Copied to clipboard
| Challenge: | Phishing often targets victims through visually perturbed texts to bypass security systems . previous studies have used synthetic datasets that do not contain real-world phishing cases . |
| Approach: | They propose a visual perturbation dataset to target phishing attacks using visual perturbations. |
| Outcome: | The proposed dataset includes real-world phishing cases annotated with visual perturbations . language models trained on the proposed dataset achieved an accuracy of approximately 96% . |
A Simple and Robust Approach to Detecting Subject-Verb Agreement Errors (N19-1)
Copied to clipboard
| Challenge: | a recent study shows that neural sequential labelers overfit their training data to detect SVA errors. |
| Approach: | They propose a simple protocol that generates a neural sequential labeler from silver standard data and gold standard data. |
| Outcome: | The proposed method leads to more robust detection of SVA errors on silver standard data and gold standard data. |
Compete to Complete: Co-opetition Adversarial Learning for Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to reduce hallucination in large language models lack a robust mechanism for generating a generative model. |
| Approach: | They propose a framework that formulates retriever–generator training in RAG as a minimax game. |
| Outcome: | The proposed framework improves retrieval-augmented generation performance on seven benchmark datasets. |
Learning to Contrast the Counterfactual Samples for Robust Visual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods of generating counterfactual samples are not fully utilized in the task of Visual Question Answering (VQA). |
| Approach: | They propose a self-supervised contrastive learning mechanism to learn the relationship between original samples, factual samples and counterfactual samples. |
| Outcome: | The proposed method surpasses state-of-the-art models on the VQA-CP dataset, a diagnostic benchmark for assessing the VQ model’s robustness. |
Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that model steering can preserve fluency and unrelated abilities, but it fails to preserve robustness specificity. |
| Approach: | They propose a framework that distinguishes three dimensions of specificity: general, control, and robustness. |
| Outcome: | The proposed framework distinguishes three dimensions of specificity: general (preserving fluency and unrelated abilities), control (preserving related control properties), and robustness (preserving control properties under distribution shifts). |
Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) evolve to autonomous agents synthesizing real-time information, but their reasoning capabilities introduce an unexpected attack surface. |
| Approach: | They propose a framework that constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions. |
| Outcome: | The proposed framework constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions. |
Challenges in Automated Debiasing for Toxic Language Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for debiasing toxic language data are limited in their ability to prevent biased behavior in toxic language detection systems. |
| Approach: | They propose to debiase toxic language detection models using lexical and dialectal markers using synthetic labels instead of traditional methods. |
| Outcome: | The proposed method reduces dialectal associations with toxicity despite the use of synthetic labels . |
Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to task-oriented dialogue systems require a large number of handcrafted features and labels. |
| Approach: | They propose a "Two-Teacher One-Student" learning framework for task-oriented dialogue . the framework amalgamates knowledge from two teacher networks and provides guidance . |
| Outcome: | The proposed framework outperforms baseline methods on two benchmark datasets . it can retrieve accurate KB entities and generate human-like responses simultaneously . |
Arabic Dataset for LLM Safeguard Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on large language models have focused on English, but the safety of LLMs in Arabic remains under-explored. |
| Approach: | They propose to use Arabic-region-specific questions to evaluate LLMs' safety . they use a dual-perspective evaluation framework to examine differences between LLM responses . |
| Outcome: | The proposed framework assesses the LLM responses from both governmental and opposition viewpoints. |
Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve robustness require changing the fine-tuning process or large-scale data augmentation, which are infeasible or cost prohibitive for closed-source models. |
| Approach: | They propose to prioritize more complex examples or replace existing training examples with LLM-generated data to improve performance on OOD NLI datasets. |
| Outcome: | The proposed methods improve performance on difficult OOD datasets while training with synthetic data leads to substantial improvements on easier OOD data. |
Document-Level Event Factuality Identification via Adversarial Neural Network (N19-1)
Copied to clipboard
| Challenge: | Document-level event factuality identification is crucial for discourse understanding in NLP . identifying document-level factual of events requires comprehensive understanding of documents . |
| Approach: | They propose to construct a corpus annotated with document- and sentence-level event factuality information on English and Chinese texts. |
| Outcome: | The proposed model outperforms baselines on the constructed corpus. |
LeakDojo: Decoding the Leakage Threats of RAG Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have failed to assess RAG leakage risks for large language models . constructing and maintaining highquality RAG knowledge databases has become increasingly costly . |
| Approach: | They propose a framework for controlled evaluation of RAG leakage using query generation and adversarial instructions. |
| Outcome: | The proposed framework compares six existing attacks across fourteen LLMs, four datasets, and diverse RAG systems. |
Increasing Robustness to Spurious Correlations using Forgettable Examples (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural NLP models often exploit spurious correlations to perform tasks. minority examples have been shown to increase the out-of-distribution generalization of pre-trained language models. |
| Approach: | They propose to use example forgetting to find minority examples without prior knowledge of spurious correlations in the dataset. |
| Outcome: | The proposed approach improves out-of-distribution generalization on minorities . it shows that minority examples are more robust on challenging datasets . |
Evaluating the Validity of Word-level Adversarial Attacks with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing adversarial examples can generate invalid adversarials due to significant changes in semantic meanings compared to their originals. |
| Approach: | They propose to use a large language model to evaluate adversarial examples by semantic constraints. |
| Outcome: | The proposed method can generate valid adversarial examples even when they are not equipped with semantic constraints. |
Merger-as-a-Stealer: Stealing Targeted PII from Aligned LLMs with Model Merging (2025.emnlp-main)
Copied to clipboard
| Challenge: | Model merging is a promising approach for updating large language models . but unmonitored mergers can introduce significant security vulnerabilities . |
| Approach: | They propose a model merging attack surface where a malicious merger can extract PII from an aligned model with model merg. |
| Outcome: | The proposed framework can extract PII from an aligned model with model merging. |
Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities (2025.naacl-long)
Copied to clipboard
Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
| Challenge: | Recent research shows that Large Language Models (LLMs) are vulnerable to automated jailbreak attacks. |
| Approach: | They propose a framework that crafts adversarial LLMs with enhanced jailbreak ability. |
| Outcome: | ADV-LLM significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. |
Know thy Corpus! Robust Methods for Digital Curation of Web corpora (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for estimating the lexicon of Web corpora have not been used to train pre-trained models. |
| Approach: | They propose a framework for digital curation of Web corpora to provide robust estimation of their parameters. |
| Outcome: | The proposed framework provides robust estimation of Web corpora's composition and lexicon . the proposed framework is similar to the BNC and ELMO models, but lacks curated categories . |
RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent named entity recognition models have great performance on many conventional benchmarks, but it is not reliable in realistic applications. |
| Approach: | They propose a method to create natural adversarial examples using Wikidata and pre-trained language models. |
| Outcome: | The proposed method produces natural adversarial examples with a shifted distribution from training data. |
AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents (2025.findings-acl)
Copied to clipboard
Guhong Chen, Liyang Fan, Zihan Gong, Nan Xie, Zixuan Li, Ziqiang Liu, Chengming Li, Qiang Qu, Hamid Alinejad-Rokny, Shiwen Ni, Min Yang
| Challenge: | Existing legal language models struggle with dynamic courtroom interactions, resulting in overfitting to standardized legal tasks. |
| Approach: | They propose a new adversarial evolutionary approach for agents that performs dynamic knowledge learning and evolution through structured adversarials in a simulated courtroom program. |
| Outcome: | The proposed approach outperforms existing LLM-based models in three critical dimensions: cognitive agility, professional knowledge, and logical rigor. |
Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Despite advances in training Large Language Models, they remain vulnerable to jailbreak, an adversarial attack method. |
| Approach: | They propose an adversarial jailbreak algorithm that exploits the gradient information of the suffix tokens to accelerate the optimization process. |
| Outcome: | The proposed model achieves 1.5x speedup while maintaining high attack success rates. |
Adversarial Learning for Discourse Rhetorical Structure Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing top-down discourse rhetorical structure parsers make local decisions and ignore global parsing. |
| Approach: | They propose a method to transform gold standard and predicted constituency trees into tree diagrams with two color channels. |
| Outcome: | The proposed method improves performance on RST-DT and CDTB corpora and can leverage global context. |
Beyond Blind Following: Evaluating Robustness of LLM Agents under Imperfect Guidance (2026.eacl-long)
Copied to clipboard
Yao Fu, Ran Qiu, Xinhe Wang, Jacob Sansom, Sathvika Ayyappa Prabhu, Huijie Tang, Jaekyeom Kim, Sungryull Sohn, Honglak Lee
| Challenge: | Large language models (LLMs) have shown strong capabilities as task-solving agents across interactive domains, but in complex environments, auxiliary guidance may be imperfect. |
| Approach: | They propose a benchmark to measure the robustness of large language models under imperfect guidance. |
| Outcome: | The proposed benchmark compared LLM agents in navigation, cooking, and gaming in a variety of environments with auxiliary guidance and noisy or underspecified instructions extracted from demonstrations. |
Whispers of Doubt Amidst Echoes of Triumph in NLP Robustness (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to measure robustness are problematic, and out-of-domain evaluations are no longer relevant. |
| Approach: | They examine models of different sizes spanning different architectural choices and pretraining objectives. |
| Outcome: | The results show that not all out-of-domain tests provide insight into robustness . merely scaling models does not make them adequately robust . |
On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models (N19-1)
Copied to clipboard
| Challenge: | Existing methods for assessing the robustness of sequence-to-sequence models have been ignored by the literature. |
| Approach: | They propose an evaluation framework for adversarial attacks on seq2seq models that takes the semantic equivalence of the pre- and post-perturbation input into account. |
| Outcome: | The proposed framework breaks the assumption that source perturbations should not result in changes in the expected output, but allows for meaning-preserving perturbations that change the output sequence. |
AICD Bench: A Challenging Benchmark for AI-Generated Code Detection (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for detecting AI-generated code are limited to binary human–machine classification under in-distribution settings. |
| Approach: | They propose to use AICD Bench to build a robust binary classification framework for large language models. |
| Outcome: | The proposed benchmark spans 2M examples, 77 models across 11 families, and 9 programming languages. |
Are AI-Generated Text Detectors Robust to Adversarial Perturbations? (2024.acl-long)
Copied to clipboard
| Challenge: | Existing detectors for AI-generated text lack robustness against adversarial perturbations, with even minor changes in characters or words causing a reversal in distinguishing between human-created and AI-generated text. |
| Approach: | They propose a siamese calibration technique to train the model to make equally confident predictions under different noise, which improves the model’s robustness against adversarial perturbations. |
| Outcome: | The proposed detector outperforms baseline methods on four datasets and is more generalizable in cross-domain, cross-genre, and mixed-source scenarios. |
When Allies Turn Foes: Exploring Group Characteristics of LLM-Based Multi-Agent Collaborative Systems Under Adversarial Attacks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the group characteristics of adversarial agents in multi-agent collaborative systems . collaborative agents are tasked with generating counterfactual answers to a given collaborative problem . |
| Approach: | They evaluate collaborative systems under adversarial attacks and propose methods to mitigate them . they also introduce a new metric to quantify the robustness of collaborative systems against such attacks . |
| Outcome: | The proposed method has been proven effective against adversarial attacks. |
To Answer or Not to Answer (TAONA): A Robust Textual Graph Understanding and Question Answering Approach (2025.findings-emnlp)
Copied to clipboard
Yuchen Yan, Aakash Kolekar, Sahika Genc, Wenju Xu, Edward W Huang, Anirudh Srinivasan, Mukesh Jain, Qi He, Hanghang Tong
| Challenge: | Existing studies assume that generated answers integrate all relevant information from the textual graph. |
| Approach: | They propose a novel GraphRAG model that integrates all relevant information from the textual graph into the generated answer. |
| Outcome: | Extensive experiments validate TAONA’s superior performance for both A-side and B-side tasks. |
SEAD: A Surrogate-free Label-only Membership Inference Attack against Pre-trained LLMs with Semantic-Aware Density (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing membership inference attacks require access to complete logits, but such access is often unavailable in real-world deployments where only the generated text is exposed. |
| Approach: | They propose a surrogate-free label-only MIA approach that directly estimates token probabilities through Monte Carlo sampling of the target model. |
| Outcome: | The proposed approach outperforms existing label-only attacks and serves as a foundational density estimator in the label-exclusive setting. |
A Reinforcement Learning Framework for Robust and Secure LLM Watermarking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing watermarking algorithms rely on heuristic green/red token lists . however, these lists are inconsistent and can be compromised . |
| Approach: | They propose a framework for robust and secure LLM watermarking using reinforcement learning. |
| Outcome: | The proposed method achieves state-of-the-art trade-off across all criteria with notable improvements in resistance to spoofing attacks without degrading other criteria. |
Exploring Robust Overfitting for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent literature has revealed their vulnerability to crafted adversarial examples on a wide range of NLP tasks. |
| Approach: | They propose to combine regularization methods with adversarial training to mitigate robust overfitting for pre-trained language models by scaling the model's loss. |
| Outcome: | The proposed methods mitigate robust overfitting upon three top adversarial training methods and further promote adversarially robustness. |
RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Existing task-oriented dialog systems are less than satisfactory in robustness evaluation . existing systems are weak in robustity evaluation based on pre-training and fine-tuning . |
| Approach: | They propose to use a set of training examples to evaluate model generalization ability . they propose to include tasks with limited training data to favor models with strong generalization abilities . |
| Outcome: | The proposed model generalizes well with limited training data and is robust to user input across domains. |
Reevaluating Adversarial Examples in Natural Language (2020.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art adversarial examples lack a common definition of what constitutes success . human surveys show that to preserve semantics, we need to increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences. |
| Approach: | They propose a unified definition of what constitutes a successful adversarial example . they propose four categories of constraints that are used to define adversarials . |
| Outcome: | The proposed framework is based on the outputs of two state-of-the-art synonym substitution attacks. |
Measuring Robustness for NLP (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to evaluate NLP models are limited to news domains and cannot be generalized to other domains. |
| Approach: | They propose a measure of NLP quality based on robustness . they measure consistency of cross-domain accuracy and introduce coefficient of variation and gamma-Robustness based upon human evaluation . |
| Outcome: | The proposed approach shows higher agreement with human evaluation than accuracy scores on ranking machine translation systems. |
Pass off Fish Eyes for Pearls: Attacking Model Selection of Pre-trained Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing feature-based model selection methods are vulnerable to fine-tuning and are not reliable indicators for the PTM’s transferability. |
| Approach: | They propose feature-based model selection methods which assess PTMs’ transferability to a specific task in a fast way without fine-tuning. |
| Outcome: | The proposed methods can make FMS mistakenly judge transferability of models and can be combined with the backdoor attack to misguide the FMS to select poisoned models. |
Red-Teaming LLM Multi-Agent Systems via Communication Attacks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Model-based Multi-Agent Systems (LLM-MAS) have revolutionized complex problem-solving capability by enabling agent collaboration through message-based communications. |
| Approach: | They propose an attack that exploits communication mechanisms in Large Language Model-based Multi-Agent Systems (LLM-MAS) by intercepting and manipulating inter-agent messages. |
| Outcome: | The proposed attack exploits communication mechanisms in large language model-based multi-agent systems by intercepting and manipulating inter-agencies. |
CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoor (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fingerprinting model ownership traces are vulnerable to illegal plagiarism and are not reliable. |
| Approach: | They propose a rule-driven fingerprinting framework that encodes contextual correlations across multiple dialogue turns. |
| Outcome: | The proposed framework achieves stronger stealth and robustness than previous work. |
RetinaQA: A Robust Knowledge Base Question Answering Model for both Answerable and Unanswerable Questions (2024.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge base question answering models assume all questions to be answerable. |
| Approach: | They propose a new KBQA model that unifies two key ideas in a single architecture . they propose logical form discrimination and sketch-filling-based construction for unanswerable questions . |
| Outcome: | The proposed model outperforms existing models in handling answerable and unanswerable questions. |
G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Model (LLM)-based Multi-agent Systems (MAS) have demonstrated remarkable capabilities in various complex tasks, but their vulnerability to adversarial attacks, misinformation propagation, and unintended behaviors have raised significant concerns. |
| Approach: | They propose a topology-guided security lens and treatment for robust LLM-MAS that leverages graph neural networks to detect anomalies on the multi-agent utterance graph and employ topological intervention for attack remediation. |
| Outcome: | Experiments show that the proposed security lens recovers 40% of the performance under various attack strategies and integrates with mainstream MAS with security guarantees. |
Jailbreaks as Inference-Time Alignment: A Framework for Understanding Safety Failures in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models are safety-aligned to prevent harmful response generation . prior work on jailbreak effectiveness has focused on analyzing success rate of jailbreaks . |
| Approach: | They propose to frame jailbreaks as inference-time alignment and draw suboptimal bounds . they also propose a Safety-Net to measure how vulnerable an LLM is to jailbreak attacks . |
| Outcome: | a new framework allows researchers to show how vulnerable an LLM is to jailbreaks . a Safety-Net measures how vulnerable the model is to attacks, the authors say . |
Do Explicit Alignments Robustly Improve Multilingual Encoders? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Explicit alignment objectives based on bitexts like Europarl and MultiUN have been shown to improve cross-lingual representations. |
| Approach: | They propose a new contrastive alignment objective that can better utilize bitexts . they propose to use a random sample of 1 million pair subset of OPUS data . |
| Outcome: | The proposed objective outperforms existing alignment objectives on a random 1 million pair subset of the OPUS dataset. |
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical research has progressed at a tremendous pace, with PubMed2 indexing well over 1M publications per year in the past eight years. |
| Approach: | They propose a semi-supervised procedure that bootstraps biomedical NLI datasets from positive entailment examples present in biomedically published texts. |
| Outcome: | The proposed procedure bootstraps biomedical NLI datasets from positive entailment examples from biomedically challenging texts. |
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that neural natural language processing models are vulnerable to backdoor attacks. |
| Approach: | They propose to inject neural models with backdoors activated by word substitution . their results raise a serious alarm to the security of NLP models, they argue . |
| Outcome: | The proposed backdoors are activated by a learnable combination of word substitution and exhibit higher invisibility than previous methods. |
CLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Evaluating the performance of Grammatical Error Correction systems is a challenging task due to its subjectivity. |
| Approach: | They propose a method to evaluate GEC systems in multi-reference evaluation setting . they use consistent edit boundaries to eliminate bias caused by inconsistent edit boundaries . |
| Outcome: | The proposed evaluation metric eliminates bias caused by inconsistent edit boundaries on six English reference sets. |
ReportLogic: Evaluating Logical Quality in Deep Research Reports (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation frameworks that evaluate large language models for Deep Research largely ignore this requirement. |
| Approach: | They propose a benchmark that quantifies report-level logical quality through a reader-centric lens of auditability. |
| Outcome: | The proposed model quantifies logical quality through a reader-centric lens of auditability. |
On the Effectiveness of Adversarial Robustness for Abuse Mitigation with Counterspeech (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work on automated counterspeech systems focused on synthetic data but rarely looked into how the public deals with abuse. |
| Approach: | They propose to curate a new dataset of abuse and replies from footballers for study of public figure abuse and use it to examine how models can handle adversarial attacks. |
| Outcome: | The proposed model is robust against adversarial attacks across domains and can handle abuse in the real world. |
Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning (2022.acl-long)
Copied to clipboard
Qin Liu, Rui Zheng, Bao Rong, Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing approaches to generating adversarial perturbations scale up the cost of training computational complexity by the number of gradient steps it takes to obtain the adversarials. |
| Approach: | They propose a flood method which aims at better generalization and a criterion to bring hyper-parameter-dependent flooding into effect with a narrowed-down search space by measuring how the gradient steps taken within one epoch affect the loss of each batch. |
| Outcome: | The proposed method improves BERT’s resistance to textual adversarial attacks by a large margin and achieves state-of-the-art robust accuracy on various text classification and GLUE tasks. |
Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretraining (2026.findings-acl)
Copied to clipboard
| Challenge: | Conformal prediction (CP) has shown promise in offering correctness guarantees for LLMs, but it faces major challenges in continual domain pretraining (CDP). |
| Approach: | They propose an adaptive rejection and non-exchangeable CP framework that allows the LLM to selectively abstain from answering when its confidence or competence shifts significantly. |
| Outcome: | Experiments show that the proposed framework improves performance under continuous domain pretraining scenarios. |
When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models exhibit severe multi-turn sycophancy in clinical dialogue . high initial diagnostic capability does not imply high belief stability . |
| Approach: | They propose a stress test framework that evaluates belief stability under escalating pressure. |
| Outcome: | The proposed stress test framework reduces the risk of multi-turn sycophancy in clinical dialogue . it eliminates belief change and improves robustness in training time . |
Multi-target Backdoor Attacks for Code Pre-trained Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work for backdoor attacks on neural code models insert triggers into task-specific data for code-related downstream tasks, limiting the scope of attacks. |
| Approach: | They propose task-agnostic backdoor attacks for code pre-trained models . they use two learning strategies to implant backdoors into code understanding and generation models - Poisoned Seq2Seq learning and token representation learning . |
| Outcome: | The proposed model is pre-trained with two learning strategies to support the multi-target attack of downstream code understanding and generation tasks. |
Contextualized Perturbation for Textual Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for generating adversarial examples are driven by local heuristic rules that are agnostic to the context, resulting in unnatural and ungrammatical outputs. |
| Approach: | They propose a ContextuaLized AdversaRial Example generation model that generates fluent and grammatical outputs through a mask-then-infill procedure. |
| Outcome: | The proposed model outperforms baseline models in terms of attack success rate, textual similarity, fluency and grammaticality. |
Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, but are vulnerable to backdoor attacks. |
| Approach: | They propose a chain-of-scrutiny approach which leverages LLMs’ unique reasoning abilities to mitigate backdoor attacks. |
| Outcome: | The proposed model is well-suited for the popular API-only LLM deployments, enabling detection at minimal cost and with little data. |
Teaching Models to Balance Resisting and Accepting Persuasion (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to persuasion, which can pose risks when faced with an adversarial interlocutor. |
| Approach: | They propose a method to balance positive and negative persuasion by using recursive dialogue trees to train models to accept persulasion. |
| Outcome: | The proposed model-based training improves resistance to misinformation and resilience to being challenged while also resulting in the best overall performance on multi-agent debates across two domains. |
Counterfactual Adversarial Learning with Representation Interpolation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models with statistical bias are prone to memorized correlations . large pre-trained models such as BERT have revolutionized the model development paradigm in natural language processing . |
| Approach: | They propose a framework to tackle the problem from a causal perspective using a latent space interpolation approach. |
| Outcome: | Extensive experiments show that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering. |
Making Language Models Robust Against Negation (2025.naacl-long)
Copied to clipboard
| Challenge: | Negation is a semantic phenomenon that alters an expression to convey the opposite meaning. |
| Approach: | They propose a self-supervised method to make language models more robust against negation by pre-training models. |
| Outcome: | The proposed task outperforms the off-the-shelf versions on nine negation-related benchmarks. |
Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing failure discovery methods rely on prior knowledge of preference attributes . Existing methods do not scale to new models or data. |
| Approach: | They propose a preference distribution agnostic procedure that uses the reward model itself to guide controlled decoding toward mis specified responses while preserving the underlying preference class. |
| Outcome: | The proposed procedure improves robustness without degrading reward quality across models. |
Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies attribute catastrophic forgetting to the corruption of the learned representations as new relations come . Continual relation extraction models suffer from catastrophic forgetting when learning new relations . |
| Approach: | They propose to use adversarial class augmentation mechanism to learn more precise representations . they propose to train the model on a sequence of tasks where two new relations are discovered . |
| Outcome: | The proposed model improves on two popular benchmarks. |
WaveDetect: Robust Framework for Machine-Generated Text Detection via Wavelet Transform (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting LLM-generated texts falter when faced with adversarial perturbations, cross-domain shifts, and the rapid temporal evolution of the foundation model. |
| Approach: | They propose a framework that reformulates text detection as a signal processing task within the time-frequency domain. |
| Outcome: | The proposed framework achieves superior accuracy and robustness against sophisticated attacks and generalization across out-of-distribution topics. |
WebIE: Faithful and Robust Information Extraction on the Web (2023.acl-long)
Copied to clipboard
| Challenge: | Existing closed IE datasets are built using Wikipedia, but they have limitations when applied to web domains. |
| Approach: | They propose to annotate 25K triples from WebIE through crowdsourcing and introduce mWebIE, a translation of the annotated set in four other languages. |
| Outcome: | The proposed model trains on 1.6M sentences from the English Common Crawl corpus and includes negative examples to better reflect the data on the web. |
UMMF: Protecting Copyright of Large Vision-Language Models through Unlearning-based Multimodal Memorization Fingerprint (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fingerprinting large vision-Language Models rely on explicit triggers, which have limitations in terms of stealthiness and robustness. |
| Approach: | They propose to use model fingerprints to verify the ownership of large vision-Language Models (LVLMs) they use implicit model fingerprinting techniques that leverage neighboring samples as implicit model . |
| Outcome: | The proposed fingerprinting technique is superior to existing methods, but has limitations in terms of stealthiness and robustness. |
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors (2025.emnlp-main)
Copied to clipboard
Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, Min Zhang
| Challenge: | Existing methods to detect large language models (LLMs) generated for plagiarism use paraphrases to rewrite them to evade detection. |
| Approach: | They propose a training-free method that effectively fools text detectors using off-the-shelf LLMs by rewriting them to evade detection. |
| Outcome: | The proposed method deceives text detectors using off-the-shelf LLMs by rewriting them to produce human-like sentences that are less discernible by detectors. |
VIGIL: Defending LLM Agents Against Tool-Stream Injection via Verify-Before-Commit (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses for indirect prompt injection are limited by static protection mechanisms . existing models prioritize injected rules due to strict alignment, whereas static protections sever the feedback loop required for adaptive reasoning. |
| Approach: | They propose a framework that shifts the paradigm from restrictive isolation to a verify-before-commit protocol. |
| Outcome: | The proposed framework outperforms state-of-the-art dynamic defenses by reducing the attack success rate by over 22% while more thandoubling utility under attack compared to static baselines. |
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using simulated experiments, we demonstrate that MT systems can be stolen even when imitation models have different input data or architectures than their target models. |
| Approach: | They propose a defense that modifies translation outputs to misdirect optimization of imitation models. |
| Outcome: | The proposed defense degrades the adversary’s BLEU score and attack success rate at some cost in the defender’s performance and inference speed. |
Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for adversarial samples are poorly applied in computer vision . however, textual adversarials are still vulnerable to small perturbations . |
| Approach: | They propose a framework to extend existing adversarial attack methods to textual adversarials by adding optimized perturbations to embedding layer and amplifying them in forward propagation process. |
| Outcome: | The proposed framework achieves better performance even using proxy gradient information and produces more fluent and grammatical adversarial samples compared to baseline methods. |
DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising (2025.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial defense methods for natural language processing still pose challenges to adversarials. |
| Approach: | They propose a novel adversarial defense method that incorporates a diffusion layer as a denoiser between the encoder and the classifier. |
| Outcome: | The proposed method improves over existing adversarial defense methods and achieves state-of-the-art performance against black-box and white-box adversarials. |
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text (D19-1)
Copied to clipboard
| Challenge: | Existing datasets for reading comprehension tasks have been used to test the generalization of natural language understanding systems. |
| Approach: | They propose a diagnostic benchmark suite to clarify key issues related to the robustness and systematicity of NLU systems. |
| Outcome: | The proposed benchmark suite clarifies key issues related to the robustness and systematicity of NLU systems. |
The Mark Fades: Adaptive Evolutionary Paraphrase-based Attack against LLM Watermarks (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing paraphrase-based watermark removal methods struggle to balance efficacy with text quality. |
| Approach: | They propose a training-free evolutionary framework that models watermark removal as a constrained multi-objective optimization problem by using a Pseudo-Log-Likelihood-guided mutation to precisely target and modify watermark-carrying tokens. |
| Outcome: | The proposed method outperforms baseline methods on a Qwen3 series watermark scheme while maintaining high semantic fidelity. |
Modeling Adversarial Attack on Pre-trained Language Models as Sequential Decision Making (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have shown strong potential in various downstream tasks. |
| Approach: | They propose to model adversarial attack task as a sequential decision-making problem where the whole attack process is sequential with two decision- making problems, i.e., word finder and word substitution. |
| Outcome: | The proposed approach achieves the highest attack success rate with a comparable modification rate and semantic similarity to attack fine-tuned BERT. |
SGT: Securing Open-Source LLMs Against Malicious Fine-tuning via Safety Guidance Trigger (2026.acl-long)
Copied to clipboard
| Challenge: | open-weight large language models increase vulnerability to malicious fine-tuning . despite these advantages, open-source LLMs increase the potential for misuse . |
| Approach: | They propose a safety guide for open-weight large language models that guides fine-tuning toward the safety manifold to preserve alignment. |
| Outcome: | The proposed safety guidance trigger significantly improves robustness against malicious fine-tuning. |
Bypassing LLM Watermarks with Color-Aware Substitutions (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect large language models (LLMs) generated text are susceptible to a large number of false positives. |
| Approach: | They propose a watermarking approach that uses color information to determine token colors and substitute green tokens with non-green ones. |
| Outcome: | The proposed method evades detection with fewer edits and removes the watermark for arbitrarily long watermarked text. |
An empirical analysis of existing systems and datasets toward general simple question answering (2020.coling-main)
Copied to clipboard
| Challenge: | evaluators of simple factoid question answering using different datasets are not able to solve SimpleQuestions. |
| Approach: | They evaluate the progress of the field toward solving simple factoid questions over a knowledge base. |
| Outcome: | The proposed model is nearly solved on the most popular dataset, but not on the robustness of existing systems. |
Sampling-based Pseudo-Likelihood for Membership Inference Attacks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are trained on large-scale web data, which makes it difficult to grasp the contribution of each text. |
| Approach: | They propose a membership-inference attack method that uses only the input text to detect leaks. |
| Outcome: | The proposed method performs on par with existing likelihood-based methods even without likelihoods. |
Close or Cloze? Assessing the Robustness of Large Language Models to Adversarial Perturbations via Word Recovery (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models implicitly recover the original text, but it is unclear when they rely on context and when they implicitly do so. |
| Approach: | They propose to use a dictionary to recover adversarial words by using a phonetic, typo, and visual attack to study word recovery performance. |
| Outcome: | The proposed model outperforms open-source models on hateful, offensive, and toxic classification tasks. |
Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training (2022.coling-1)
Copied to clipboard
Zhongjian Miao, Xiang Li, Liyan Kang, Wen Zhang, Chulun Zhou, Yidong Chen, Bin Wang, Min Zhang, Jinsong Su
| Challenge: | Existing methods on robust neural machine translation (NMT) construct adversarial examples by injecting noise into authentic examples and indiscriminately exploit two types of examples. |
| Approach: | They propose an iterative scheduled data-switch training framework to mitigate this problem by injecting noise into authentic examples and indiscriminately exploiting two types of examples. |
| Outcome: | The proposed model outperforms several competitive benchmarks on four translation benchmarks. |
Are Language Models Agnostic to Linguistically Grounded Perturbations? A Case Study of Indic Languages (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies do not focus on linguistically grounded attacks, but pre-trained models are susceptible to these perturbations. |
| Approach: | They propose to examine whether pre-trained language models are agnostic to linguistically grounded attacks . they find that PLMs are less susceptible to linguistic perturbations than non-linguistic ones . |
| Outcome: | The proposed model is agnostic to linguistically grounded attacks, but is less susceptible to linguist attacks than non-linguistic models. |
BadWindtunnel: Defending Backdoor in High-noise Simulated Training with Confidence Variance (2025.findings-acl)
Copied to clipboard
| Challenge: | Current backdoor attack defenders in NLP typically involve data reduction or model pruning, risking losing crucial information. |
| Approach: | They propose a backdoor defender that allows precise control over training conditions to model backdoor learning behavior without affecting the final model. |
| Outcome: | The proposed model reduces the backdoor learning behavior without affecting the final model. |
Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification (D19-1)
Copied to clipboard
| Challenge: | Existing studies on adversarial attacks on deep learning models focus on generation of adversarials and defense against adversarial attacks. |
| Approach: | They propose a framework to identify and adjust malicious perturbations and block adversarial attacks for machine learning models. |
| Outcome: | The proposed framework outperforms baseline methods in blocking adversarial attacks for text classification models. |
Simple Entity-Centric Questions Challenge Dense Retrievers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain question answering has exploded in popularity due to the success of dense retrieval models. |
| Approach: | They construct a set of simple, entity-rich questions based on facts from Wikidata and test their models against supervised datasets. |
| Outcome: | The proposed model outperforms sparse retrieval methods on open-domain question answering datasets by a large margin. |
Towards automatically generating Questions under Discussion to link information and discourse structure (2020.coling-main)
Copied to clipboard
| Challenge: | Questions under Discussion (QUD) are emerging as a useful approach to spelling out the connection between information structure of sentences and nature of discourse. |
| Approach: | They propose a framework for QUD annotation based on explicit pragmatic principles . they propose generating all potentially relevant questions for a given sentence . |
| Outcome: | The proposed framework supports more reliable discourse structure annotation based on explicit questions . but the proposed approach is not robust enough for authentic data . |
Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to estimate question difficulty rely on readability formulas, retrieval-based signals, or popularity statistics. |
| Approach: | They propose a method that estimates question difficulty by computing the entropy of plausibility scores over candidate answers. |
| Outcome: | The proposed method outperforms baselines across four QA datasets and shows strong robustness across hyperparameter variations and question types. |
SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate a subset of potential conditions, missing others entirely. |
| Approach: | a new benchmark suite evaluates speaker verification models under a variety of stressors . a san francisco-based team evaluates models under natural and background conditions . |
| Outcome: | a new benchmark suite evaluates speaker verification models under stressors under a variety of conditions . the results show that some models perform better under stress conditions than others . |
Enhance Robustness of Language Models against Variation Attack through Graph Integration (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are used in many NLP applications but their vulnerability to adversarial attacks can lead to false or misleading information being distributed. |
| Approach: | They propose a method to incorporate a Chinese character variation graph into pre-trained language models to increase their robustness against character variation attacks in Chinese content. |
| Outcome: | The proposed method outperforms existing language models in combating adversarial attacks in Chinese content. |
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents (2026.acl-long)
Copied to clipboard
| Challenge: | a fundamental pillar of trustworthiness is calibration, which refers to an agent’s ability to express confidence that reliably reflects its actual performance. |
| Approach: | They propose a reinforcement learning framework that jointly optimizes task accuracy and calibration, supported by a holistic benchmark of reward designs. |
| Outcome: | The proposed framework improves calibration across tool types and shows that trained agents achieve superior calibration and exhibit robust generalization from local training environments to noisy web settings and to distinct domains such as mathematical reasoning. |
Text Style Transferring via Adversarial Masking and Styled Filling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for text style transfer suffer from two challenges: the word masking procedure may mistakenly remove unexpected words and the selected words in the word filling procedure lack diversity and semantic consistency. |
| Approach: | They propose a style transfer model with adversarial masking and styled filling techniques to solve these challenges. |
| Outcome: | The proposed model performs well on two benchmark text style transfer data sets. |
Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to attack transformer models are not effective at character level, but they are a natural attack scenario. |
| Approach: | They propose a character-level adversarial attack method against transformer models . they use a gradient-based method to find the most vulnerable words in a sentence . |
| Outcome: | The proposed method outperforms previous methods on sentence-level and token-level tasks. |
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach (2021.emnlp-main)
Copied to clipboard
Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
| Challenge: | Existing approaches to adversarial regularization treat adversarials and defending players equally, which is undesirable because only the defending player contributes to the generalization performance. |
| Approach: | They propose a method which formulates adversarial regularization as a Stackelberg game and induces a competition between a leader and a follower. |
| Outcome: | The proposed method outperforms existing adversarial regularization baselines on a set of machine translation and natural language understanding tasks. |
Enhancing Factual Consistency in Text Summarization via Counterfactual Debiasing (2025.coling-main)
Copied to clipboard
| Challenge: | Abstractive text summarization has produced fluent and informative outputs, but factual inconsistency is a challenge. |
| Approach: | They propose a framework that mitigates the causal effects of language bias and irrelevancy bias by counterfactual estimation. |
| Outcome: | The proposed framework outperforms baseline methods on two widely used summarization datasets. |
Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit substantial capabilities yet face challenges such as hallucination, outdated knowledge, and untraceable reasoning processes. |
| Approach: | They propose a retrieval-augmented generation approach that leverages adaptive adversarial training to dynamically adjust the model’s training process in response to retrieval noises. |
| Outcome: | The proposed approach improves the performance of the LLaMA-2 7B model under diverse noise conditions. |
Word-level Textual Adversarial Attacking as Combinatorial Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word-level attack models are far from perfect because of unsuitable search space reduction methods and inefficient optimization algorithms. |
| Approach: | They propose a novel adversarial adversarialist model that incorporates word substitution and particle swarm optimization to solve two problems separately. |
| Outcome: | The proposed model achieves much higher success rates and crafts more high-quality adversarial examples as compared to baseline methods. |
Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems (2022.emnlp-main)
Copied to clipboard
| Challenge: | Numerical Question Answering is the task of answering questions that require numerical capabilities. |
| Approach: | They propose to conduct numerical capability diagnosis on a series of Numerical Question Answering systems and datasets. |
| Outcome: | The proposed approach relieves existing systems’ lack of robust numerical capabilities. |
Analysing the Safety Pitfalls of Steering Vectors (2026.findings-acl)
Copied to clipboard
| Challenge: | Activation steering has emerged as a powerful tool to shape LLM behaviour without the need for weight updates. |
| Approach: | They propose to audit steering vectors obtained with Contrastive Activation Addition (CAA) and propose a mechanistic explanation for this finding. |
| Outcome: | The proposed approach significantly improves the success rate of jailbreak attacks, with stronger amplification under simple template-based attacks. |
Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multistep question answering have shown promise in generating multistep solutions, but they lack robustness. |
| Approach: | They propose a framework that trains a model to robustly answer multistep questions by generating and answering sub-questions. |
| Outcome: | The proposed framework outperforms neuro-symbolic methods on a DROP contrast set and GPT-3.5 on QA adversarial sets. |
Generating Fluent Adversarial Examples for Natural Languages (P19-1)
Copied to clipboard
| Challenge: | Current methods for building adversarial attackers for NLP are inefficient as the gradient is discarded. |
| Approach: | They propose an adversarial attacker which performs Metropolis-Hastings sampling with the guidance of gradients to solve these problems. |
| Outcome: | The proposed algorithm outperforms the baseline model on attacking capability on IMDB and SNLI. |
Defending against Insertion-based Textual Backdoor Attacks via Attribution (2023.findings-acl)
Copied to clipboard
| Challenge: | Textual backdoor attacks are vulnerable to backdoors and can be used to infect models trained on poisoned data. |
| Approach: | They propose an efficient attribution-based pipeline to defend against two insertion-based poisoning attacks, BadNL and InSent. |
| Outcome: | The proposed method can generalize sufficiently well in two common attack scenarios, which consistently improves previous methods. |
Best Student Forcing: A Simple Training Mechanism in Adversarial Language Generation (2020.lrec-1)
Copied to clipboard
| Challenge: | Language models trained with Maximum Likelihood Estimation (MLE) have been considered as a mainstream solution in Natural Language Generation (NLG) however, they are reportedly suffering from training instability and mode collapse, and therefore outperform conventional MLE models. |
| Approach: | They propose a method to improve Generative Adversarial Nets (GANs) using best student forcing and discriminators to increase training stability and sample diversity. |
| Outcome: | The proposed techniques outperform MLE models and outperformed existing approaches in terms of sample diversity and training stability. |
Stimulate the Critical Thinking of LLMs via Debiasing Discussion (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that large language models (LLMs) are often prone to stance homogeneity and human preference biases when faced with conflicting perspectives. |
| Approach: | They propose a novel two-stage training framework to address stance homogeneity bias and human preference bias by generating multi-model discussion datasets and optimizing reinforcement learning from human feedback to align with discussion correctness. |
| Outcome: | The proposed framework reduces stance homogeneity bias and human preference bias and improves generalization capabilities on non-discussion scenarios and out-of-domain datasets. |
Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors (2024.acl-long)
Copied to clipboard
| Challenge: | Recent research indicates that AI-text detection systems lack robustness and struggle to effectively differentiate perturbed texts. |
| Approach: | They propose to evaluate the robustness of current detection systems by using black-box text perturbation methods and adversarial learning experiments. |
| Outcome: | The proposed methods assess the robustness of current detection models across perturbation granularities and the impact of perturbation data augmentation on the robustity of AI-text detectors. |
Using Adversarial Examples in Natural Language Processing (L18-1)
Copied to clipboard
| Challenge: | Recent advances in machine learning have led to the use of adversarial examples in training of neural networks. |
| Approach: | They investigate the effect of using adversarial examples during training of recurrent neural networks whose text input is in the form of a sequence of word/character embeddings. |
| Outcome: | The proposed method provides regularization effect and enables training of models with greater number of parameters without overfitting. |
All That Glitters is Not Gold: Improving Robust Retrieval-Augmented Language Models with Fact-Centric Preference Alignment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn adaptive retrieval for noisy documents lack prior filtering and may lead to the loss of crucial information. |
| Approach: | They propose a method to improve retrieval performance without prior filtering . they use LLMs self-generated synthetic data as training data without manual annotation . |
| Outcome: | The proposed method performs positive document mining based on factual consistency and uses LLMs self-generated synthetic data as training data without manual annotation. |
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)
Copied to clipboard
| Challenge: | Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension. |
| Approach: | They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts . |
| Outcome: | The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials . |
Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech (2025.naacl-long)
Copied to clipboard
Eric Battenberg, RJ Skerry-Ryan, Daisy Stanton, Soroosh Mariooryad, Matt Shannon, Julian Salazar, David Teh-Hwa Kao
| Challenge: | Autoregressive (AR) Transformer-based sequence models have difficulty generalizing to sequences longer than those seen during training. |
| Approach: | They propose a system that provides cross-attention operations with relative location information. |
| Outcome: | The proposed system matches the naturalness and expressiveness of a baseline T5-based system while eliminating problems with repeated or dropped words. |
RedHerring Attack: Testing the Reliability of Attack Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | In response to adversarial text attacks, attack detection models have been proposed and shown to successfully identify text modified by adversaries. |
| Approach: | They propose to modify a text to cause the detection model to predict an attack while keeping the classifier correct. |
| Outcome: | The proposed model drops detection accuracy between 20 - 71 points while maintaining (or improving) classifier accuracy. |
SSR: Utilizing Simplified Stance Reasoning Process for Robust Stance Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for stance detection are task-agnostic, which fail to utilize task knowledge to better discriminate between genuine and bias features. |
| Approach: | They propose to incorporate stance reasoning process as task knowledge to aid in learning genuine features without using targets. |
| Outcome: | The proposed model achieves better performance than previous task-agnostic debiasing methods on new test sets. |
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing safety mechanisms for Large Language Models (LLMs) are inadequate to protect against jailbreak attacks, resulting in performance degradation on general tasks. |
| Approach: | They propose a method that directly updates a minimal set of relevant parameters to neutralize harmful behaviors while preserving the model’s utility. |
| Outcome: | The proposed model outperforms baseline methods in mitigating jailbreak attacks while preserving the model’s utility. |
P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Defending Large Language Models (LLMs) against backdoors has long been trapped in a "cat-and-mouse" dilemma where defenders passively react to ever-shifting attack strategies. |
| Approach: | They propose a general and effective defense algorithm that implants benign triggers to reshape the model’s decision boundary. |
| Outcome: | The proposed defense algorithm can neutralize malicious backdoors while preserving task performance. |
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)
Copied to clipboard
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
| Challenge: | Existing models of robustness evaluation are incomprehensive, impractical, and invalid . |
| Approach: | They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks. |
| Outcome: | The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol. |
SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods define important nodes as important and target them for attacks if the model treats nodes’ predictive influence more uniformly . Existing approaches target high predictive influence nodes but are vulnerable to malicious message injection attacks. |
| Approach: | They propose a defense mechanism that encourages the model to learn graph representations where nodes with varying importance have a more uniform influence on predictions. |
| Outcome: | Extensive experiments on the Twitter and Weibo datasets show that similarizing the predictive Influence of nodes with Contrastive Learning significantly enhances resistance against LLM-driven message injection attacks. |
Learning Rhyming Constraints using Structured Adversaries (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to text generation fail to capture higher-level structure in text, for example, rhyming patterns. |
| Approach: | They propose a method that uses a structured discriminator to learn rhyming constraints from poetry . the discriminator compares two English poetry datasets based on a learned similarity matrix . |
| Outcome: | The proposed method can learn rhyming patterns in English poetry without explicit phonetic information. |
Typographic Attacks in a Multi-Image Setting (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models are susceptible to typographic attacks, which are misclassifications caused by an attack text that is added to an image. |
| Approach: | They propose a multi-image setting for studying typographic attacks by leveraging the difficulty of the target image, the strength of the attack text, and text-image similarity. |
| Outcome: | The proposed approach improves success rates by 21% over random, non-specific methods on the CLIP model while maintaining stealth in a multi-image scenario. |
Locket: Robust Feature-Locking Technique for Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing FLoTEs (e.g., password-locked models) fail to meet these criteria. |
| Approach: | They propose a feature-locking technique that is effective in refusing locked features . they propose scalable FLoTEs that can selectively disable specific features of a model . |
| Outcome: | The proposed solution is effective (100% refusal rate), utility-preserving ( 7% utility degradation), robust (5% attack success rate), and scalable to multiple features and clients. |
Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can alter models' predicted sentiment due to their sensitivity to specific word choices. |
| Approach: | They propose a regularization technique to improve NLP model robustness by reducing the impact of input perturbations on model outputs. |
| Outcome: | The proposed method outperforms state-of-the-art methods in adversarial defense. |
Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing fine-tuning algorithms for vision-language models are restricted by patient privacy concerns and can contain imperceptible noise. |
| Approach: | They propose a framework to mitigate adversarial noise and mitigate upstream noise during fine-tuning. |
| Outcome: | The proposed framework improves model robustness and transferability while decreasing noise levels negatively impact downstream performance. |
PanicToCalm: A Proactive Counseling Agent for Panic Attacks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for training such models are limited due to ethical and logistical issues. |
| Approach: | They propose a dataset that includes high-distress episodes constructed from first-person narratives and structured around the principles of Psychological First Aid. |
| Outcome: | The proposed model outperforms baseline models in counselor-side metrics and client affect improvement. |
RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that perform deductive reasoning on inputs containing rules and statements in the English natural language do not perform consistently on the RobustLR test set. |
| Approach: | They propose a diagnostic benchmark that evaluates the robustness of language models to minimal logical edits in inputs and different logical equivalence conditions. |
| Outcome: | The proposed models do not perform consistently on the RobustLR test set. |
MoRoVoc: A Large Dataset for Geographical Variation Identification of the Spoken Romanian Language (2025.findings-emnlp)
Copied to clipboard
Andrei-Marius Avram, Bănescu Ema-Ioana, Anda-Teodora Robea, Dumitru-Clementin Cercel, Mihaela-Claudia Cercel
| Challenge: | MoRoVoc is the largest dataset for analyzing the regional variation of spoken Romanian . it has more than 93 hours of audio and 88,192 audio samples . |
| Approach: | They propose a multi-target adversarial training framework that incorporates demographic attributes as adversarials for speech models. |
| Outcome: | The proposed model achieves 78.21% accuracy for variation identification of spoken Romanian using gender as an adversarial target. |
Evaluating and Improving Factuality in Multimodal Abstractive Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current factuality metrics do not account for vision modality, thus are not adequate for vision-and-language summarization. |
| Approach: | They propose a weighted combination of CLIPScore and BERTScore to evaluate factuality for abstractive document summarization. |
| Outcome: | The proposed metric outperforms existing factuality metrics on four factuity metric-evaluation benchmarks and is robust to human judgments. |
RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings. |
| Approach: | They propose a robust RAG framework for large language models via Margin-aware Preference Optimization to enhance the accuracy and reliability of SLMs. |
| Outcome: | The proposed framework surpasses state-of-the-art benchmarks on three open-domain question answering tasks. |
Virtual Context Enhancing Jailbreak Attacks with Special Token Injection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing jailbreak attacks target the two phases of user interaction: prompt input and model computation. |
| Approach: | They propose a new tool that leverages special tokens to improve jailbreak attacks . they found that the tool can increase success rates of existing jailbreak methods by 40% . |
| Outcome: | The proposed solution can improve success rates of four widely used jailbreak methods by approximately 40% across various LLMs. |
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |
MalURLBench: A Benchmark Evaluating Agents’ Vulnerabilities When Processing Web URLs (2026.findings-acl)
Copied to clipboard
Dezhang Kong, Zhuxi Wu, Shiqi Liu, ZhiCheng Tan, Kuichen Lu, Minghao Li, Qichen Liu, Shengyu Chu, Zhenhua Xu, Xuan Liu, Meng Han
| Challenge: | Existing models struggle to detect elaborately disguised malicious URLs, despite their ability to process malicious URL's. |
| Approach: | They propose a benchmark to evaluate LLMs’ vulnerabilities to malicious URLs and a lightweight defense module to mitigate the vulnerability. |
| Outcome: | The proposed framework analyzes 61,845 attack instances spanning 10 real-world scenarios and 7 categories of real malicious websites. |
Attention-Enhancing Backdoor Attacks Against BERT-based Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights. |
| Approach: | They propose a Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating attention patterns. |
| Outcome: | The proposed method improves the effectiveness of the backdoor attacks on different backbone models and tasks. |
Defending Against Social Engineering Attacks in the Age of LLMs (2024.emnlp-main)
Copied to clipboard
Lin Ai, Tharindu Kumarage, Amrita Bhattacharjee, Zizhou Liu, Zheng Hui, Michael Davinroy, James Cook, Laura Cassani, Kirill Trapeznikov, Matthias Kirchner, Arslan Basharat, Anthony Hoogs, Joshua Garland, Huan Liu, Julia Hirschberg
| Challenge: | Existing research has developed frameworks to understand human-to-human CSE attacks. |
| Approach: | They propose a modular defense pipeline that improves detection at both the message and conversation levels. |
| Outcome: | The proposed model can be exploited to facilitate chat-based social engineering attacks and generate high-quality CSE content, but their detection capabilities are suboptimal, leading to increased operational costs for defense. |
Debiasing the Fine-Grained Classification Task in LLMs with Bias-Aware PEFT (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to mitigate label biases such as retraining, post-hoc adjustment, and parameter-efficient fine-tuning fail to address prediction propensity and discriminative ability biase. |
| Approach: | They propose a bias-aware optimization framework that incorporates two distinct label balance constraints with a PEFT strategy targeting an intermediate layer to mitigate this issue. |
| Outcome: | The proposed approach outperforms or matches the performance of full-parameter fine-tuning and LoRA, achieving superior results with lower perplexity. |
The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on corpus poisoning, but there are no studies on adversarial attacks on RAG systems. |
| Approach: | They propose a novel imperceptible retrieve-to-generate attack against RAG systems . they propose regenerative reinforcement learning framework that tracks interactions between attacker and target RAG . |
| Outcome: | The proposed framework outperforms existing attacks on factual and non-factual RAG systems with small imperceptible text perturbations. |
PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for task completion in real-world situations. |
| Approach: | They propose a PowerPoint Task Completion-Robustness (PPTC-R) benchmark to measure LLMs’ robustness to the user PPT task instruction and software version (Powerpoint). |
| Outcome: | The proposed benchmark compares 3 closed-source and 4 open-source LLMs to the PowerPoint task instruction and software version (Powerpoint) . |
Towards Robust ESG Analysis Against Greenwashing Risks: Aspect-Action Analysis with Cross-Category Generalization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing NLP methods lack robustness against greenwashed ESG content . existing methods often extract insights that reflect misleading or exaggerated sustainability claims rather than objective ESG performance. |
| Approach: | They propose to use a dataset to improve the robustness of ESG analysis amid the prevalence of greenwashing to analyze sustainability reports. |
| Outcome: | The proposed model improves robustness against greenwashed claims rather than objective ESG performance. |
How Robust Are the QA Models for Hybrid Scientific Tabular Data? A Study Using Customized Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing tabular QA models are lacking in understanding their robustness on scientific information. |
| Approach: | They propose a dataset to assess the robustness of tabular QA models on scientific hybrid tabular data. |
| Outcome: | The proposed model performs well on scientific tables and text, while the best score is 0.462. |
Neural Topic Modeling with Cycle-Consistent Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances on deep generative models have attracted significant interest in neural topic modeling. |
| Approach: | They propose an adversarial-neural topic model which uses Dirichlet prior to capture the semantic patterns in latent topics. |
| Outcome: | The proposed models outperform competing models on unsupervised/supervised topic modeling and text classification. |
Debias NLU Datasets via Training-free Perturbations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to debiase NLU models capture biased features that are independent of the task but spuriously correlated to labels. |
| Approach: | They propose a framework that conducts training-free perturbations on samples containing biased features to Debias NLU Datasets. |
| Outcome: | The proposed framework shows competitive performance with previous state-of-the-art debiasing strategies. |
Spatiotemporal Sycophancy: Negation-Based Gaslighting in Video Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Vid-LLMs lack robust mechanisms for maintaining grounded spatiotemporal beliefs under conversational feedback. |
| Approach: | They propose a negation-based gaslighting evaluation framework and introduce a benchmark to investigate spatiotemporal sycophancy. |
| Outcome: | The proposed framework evaluates state-of-the-art Vid-LLMs across video understanding tasks. |
Adversary-Aware DPO: Enhancing Safety Alignment in Vision Language Models via Adversarial Training (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Adversary-aware DPO (ADPO) is a training framework that explicitly considers adversary. |
| Approach: | a new framework integrates adversarial training into a pre-trained large language model to enhance safety alignment . adversary-aware DPO provides a framework that explicitly considers adversary . |
| Outcome: | a new training framework outperforms baselines in safety alignment and general utility of large language models. |
Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models rely heavily on idiosyncratic biases of datasets. |
| Approach: | They propose a method which discourages models from exploiting biases while enabling them to receive enough incentive to learn from all the training examples. |
| Outcome: | The proposed method improves on out-of-distribution datasets while maintaining original in-district accuracy. |
Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP (2022.emnlp-main)
Copied to clipboard
| Challenge: | Textual adversarial samples are often misrepresented in research on security, evaluation, explainability, and data augmentation. |
| Approach: | They propose to use adversarial samples to evaluate their methods on security tasks to demonstrate the real-world concerns rather than developing impractical methods. |
| Outcome: | The proposed method has higher practical value than the current benchmark. |
"Penny Wise, Pixel Foolish": Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations (2026.findings-acl)
Copied to clipboard
| Challenge: | Mobile Agents are a key component of the “Agentic Economy” where they perform high-stakes financial transactions. |
| Approach: | They propose a systemic vulnerability termed Visual Dominance Hallucination (VDH) VDH exploits the modality gap in CLIP-based encoders via a novel Semantic-Decoupling Loss. |
| Outcome: | The proposed framework exploits the modality gap in CLIP-based encoders by preserving fidelity. |
Weed Out, Then Harvest: Dual Low-Rank Adaptation is an Effective Noisy Label Detector for Noise-Robust Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that PEFT can fine-tune language models without relying on perfectly labeled datasets. |
| Approach: | They propose a framework that decouples sample selection from model training by introducing clean and noisy LoRA. |
| Outcome: | The proposed framework decouples sample selection from model training. |
UPTON: Preventing Authorship Leakage from Public Text Release via Data Poisoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent authorship attribution models can reveal the true authorship of unseen texts with high accuracies, with some cases up to 95% accuracy. |
| Approach: | They propose a solution that weakens authorship features in training samples and makes released texts unlearnable by exploiting black-box data poisoning methods. |
| Outcome: | The proposed model weakens authorship features in training samples and makes released texts unlearnable. |
A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that fine-tuned textual transformer models are vulnerable to adversarial text perturbations. |
| Approach: | They extract 13 different features representing a wide range of input fine-tuning corpora properties and use them to predict adversarial robustness of the fine- tuned models. |
| Outcome: | The proposed framework can be used as an additional tool for robustness evaluation since it saves 30x-193x runtime compared to the traditional technique and can be easily used under adversarial training. |
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective (2023.findings-acl)
Copied to clipboard
Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang
| Challenge: | Pre-trained language models (PLMs) have improved generalization performance but the out-of-distribution (OOD) generalization problem remains a challenge in many NLP tasks. |
| Approach: | They propose to create a benchmark for evaluating out-of-distribution (OOD) generalization in NLP models. |
| Outcome: | The proposed benchmarks highlight the importance of OOD robustness and provide insights on how to measure it and improve it. |
DiSec: Mitigating Backdoors in Pre-trained Language Models via Disentanglement of Adversarial Weights for Secure Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defenses rely on privileged assumptions, limiting their applicability in realistic settings. |
| Approach: | They propose a task-agnostic backdoor attack that contaminates pre-trained language models . authors propose auxiliary text purification framework that uses only clean auxiliary data . |
| Outcome: | The proposed framework suppresses attack success while preserving clean-task utility. |
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial models rely on keyword matching and ignore relevant contextual relations for answer prediction. |
| Approach: | They propose to use keyword matching to attack model with two biases that rely on a perturbed answer sentence and a distracting answer sentence to misguide model. |
| Outcome: | The proposed method produces fluent and grammatical adversarial contexts while maintaining gold answers. |
MeetingQA: Extractive Question-Answering on Meeting Transcripts (2023.acl-long)
Copied to clipboard
| Challenge: | Meeting transcripts are a promising domain for natural language tasks . lack of annotated data impedes research on other important tasks in this domain . |
| Approach: | They propose an extractive QA dataset comprising questions asked by meeting participants and corresponding responses. |
| Outcome: | The proposed dataset extracts questions asked by meeting participants and corresponding responses from transcripts. |
Can Post-Training Transform LLMs into Causal Reasoners? (2026.findings-acl)
Copied to clipboard
| Challenge: | Causal inference is a core component of human cognition and requires decision-makers to distinguish between causation and association. |
| Approach: | They propose a dataset comprising seven core causal tasks for training and five diverse test sets and evaluate five different post-training approaches. |
| Outcome: | The proposed model achieves 93.5% accuracy on the CaLM benchmark, compared to 55.4% by OpenAI o3. |
Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment (2025.emnlp-main)
Copied to clipboard
Pedram Zaree, Md Abdullah Al Mamun, Quazi Mishkatul Alam, Yue Dong, Ihsen Alouani, Nael Abu-Ghazaleh
| Challenge: | Recent research has shown that carefully crafted jailbreak inputs can induce large language models to produce harmful outputs, despite safety measures such as alignment. |
| Approach: | They propose a method for generating highly effective Jailbreak attacks that selectively strengthen or weaken attention among different parts of the prompt. |
| Outcome: | The proposed attacks amplify the success rate of existing Jailbreak algorithms while lowering generation cost. |
A Troublemaker with Contagious Jailbreak Makes Chaos in Honest Towns (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research focuses on single-agent attacks and shared memory attacks, but real-world scenarios often involve independent memory. |
| Approach: | They propose a large-scale, multi-agent, multitopology attack evaluation framework that exploits the memory of an agent to make it more vulnerable to jailbreak attacks. |
| Outcome: | The proposed framework improves on the troublemaker makes chaos in Honest Town task with 23.51%, 18.95%, and 52.93% improvements in line, star topologies, and 100-agent settings. |
Benchmarking and Improving LLM Robustness for Personalized Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations focus on whether a model’s responses align with a user’s preferences, but factuality is an important yet overlooked dimension. |
| Approach: | They propose a scalable framework for evaluating robustness of large language models in personalization and a new dataset, PERGData. |
| Outcome: | The proposed framework improves robustness by 25% across models. |
Robustness of Learning from Task Instructions (2023.findings-acl)
Copied to clipboard
| Challenge: | traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples. |
| Approach: | a new study investigates the system robustness when instructions are manipulated and paraphrased . task instructions give the model the definition of the task and allow it to output the appropriate answer . |
| Outcome: | a new study shows that supervised learning is robust when instructions are manipulated, paraphrased or iii from different levels of conciseness. |
Here’s a Free Lunch: Sanitizing Backdoored Models with Model Merge (2024.findings-acl)
Copied to clipboard
| Challenge: | democratization of pre-trained language models brings significant security risks, including backdoor attacks. |
| Approach: | They propose to merge a backdoored model with other homogeneous models to remediate backdoor vulnerabilities. |
| Outcome: | The proposed model merging approach outperforms other models on classification tasks without additional resources or specific knowledge. |
How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of Large Language Models for clinical numerical reasoning provide limited operation-level coverage and limited robustness of numerical understanding across clinical note formats. |
| Approach: | They propose a benchmarking tool that evaluates four main types of clinical numeracy . they present longitudinal MIMIC-IV vital-sign records in three semantically equivalent representations . |
| Outcome: | The proposed benchmark evaluates four main types of clinical numeracy: value retrieval, arithmetic computation, relational comparison, and aggregation. |
Assessing the Sensitivity and Alignment of FOL Closeness Metrics (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent paradigm of logical reasoning with tool-augmented large language models has been successful, but the correctness of FOL statements often go unverified due to the lack of a reliable evaluation metric. |
| Approach: | They compare sensitivity of existing metrics and their alignment with LLM to evaluate FOLs. |
| Outcome: | The proposed metrics improve the sensitivity and robustness of the proposed models. |
Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models (2025.findings-emnlp)
Copied to clipboard
Zonghao Ying, Deyue Zhang, Zonglei Jing, Yisong Xiao, Quanchen Zou, Aishan Liu, Siyuan Liang, Xiangzheng Zhang, Xianglong Liu, Dacheng Tao
| Challenge: | Existing methods for implementing multi-turn jailbreaks struggle to balance semantic coherence with attack effectiveness, resulting in benign semantic drift or ineffective detection evasion. |
| Approach: | They propose a framework that reformulates harmful queries into benign reasoning tasks and leverages LLMs’ strong reasoning capabilities to compromise safety alignment. |
| Outcome: | The proposed framework achieves state-of-the-art attack effectiveness in complex conversational scenarios, with average ASRs increasing by up to 96%. |
Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable capabilities but are vulnerable to adversarial “jailbreak” attacks designed to bypass safety guardrails. |
| Approach: | They propose to empower a large language model to be its own red teamer . safety self-play allows the model to act as both the Attacker and Defender . |
| Outcome: | The proposed approach outperforms baselines trained on static adversarial datasets and establishes a new benchmark for proactive safety alignment. |
Multi-trait User Simulation with Adaptive Decoding for Conversational Task Assistants (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to model conversational traits are costly and time consuming. |
| Approach: | They propose a method that generates diverse user profiles at decoding-time by sampling from trait-specific Language Models. |
| Outcome: | The proposed method generates diverse user profiles at decoding-time without fine-tuning. |
Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly employed as automated evaluators to assess the safety of generated content. |
| Approach: | They evaluate 11 LLM judge models across critical safety domains . apologetic language artifacts alone can skew evaluator preferences by up to 98% . |
| Outcome: | The findings show that apologetic language artifacts alone can skew evaluator preferences by up to 98% . |
AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science (2025.findings-emnlp)
Copied to clipboard
An Luo, Xun Xian, Jin Du, Fangqiao Tian, Ganghua Wang, Ming Zhong, Shengchun Zhao, Xuan Bi, Zirui Liu, Jiawei Zhou, Jayanth Srinivasa, Ashish Kundu, Charles Fleming, Mingyi Hong, Jie Ding
| Challenge: | Large language models (LLMs) have advanced the automation of data science workflows, yet it remains unclear whether they can critically leverage external domain knowledge as human data scientists do in practice. |
| Approach: | They propose a benchmark to evaluate how large language models handle external domain knowledge in tabular prediction tasks. |
| Outcome: | The proposed model evaluates whether it can critically leverage external domain knowledge as human data scientists do in practice. |
Intrinsic Test of Unlearning Using Parametric Knowledge Traces (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for “unlearning” information captured in large language models rely on behavioral tests without monitoring residual knowledge in model parameters. |
| Approach: | They propose a general evaluation methodology that uses vocabulary projections to inspect concepts encoded in model parameters. |
| Outcome: | The proposed method detects changes in parametric traces of unlearned concepts and localizes them in two open-source LLMs. |
Rethinking LLM Watermark Detection in Black-Box Settings: A Non-Intrusive Third-Party Framework (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing secret-key schemes tightly couple detection with injection . this dependency creates a fundamental barrier for real-world governance . |
| Approach: | et al. introduce a black-box framework for non-intrusive, third-party watermark verification . they propose a proxy model to amplify watermark-relevant signals and complementary relative measurements . |
| Outcome: | a new framework decouples detection from injection and assesses alignment of query text with watermark distributions. |
Evaluating and Enhancing the Robustness of Code Pre-trained Models through Structure-Aware Adversarial Samples Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained code models have made significant strides in the field of neural code intelligence, but they are susceptible to adversarial attacks that subtly modify the input sequence and can impair generalization. |
| Approach: | They propose a set of novel robustness evaluation methods based on the intrinsic structure of the code to explore the impact of imperceptible perturbation. |
| Outcome: | The proposed methods have demonstrated their effectiveness across a wide range of models and tasks, and are able to predict the performance of perturbed models. |
CROWD: Certified Robustness via Weight Distribution for Smoothed Classifiers against Backdoor Attack (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on the certification of robustness of NLP models against backdoor attacks have focused on empirical defences against adversarial attacks without formal guarantees. |
| Approach: | They propose a model-agnostic mechanism for large-scale models that applies to complex model structures without the need for assessing model architecture or internal knowledge. |
| Outcome: | The proposed model-agnostic mechanism is tested on a diverse range of datasets and tasks, showing it can be used to mitigat backdoor triggers. |
Core: Robust Factual Precision with Informative Sub-Claim Identification (2025.findings-acl)
Copied to clipboard
Zhengping Jiang, Jingyu Zhang, Nathaniel Weir, Seth Ebner, Miriam Wanner, Kate Sanders, Daniel Khashabi, Anqi Liu, Benjamin Van Durme
| Challenge: | Using the Decompose-Then-Verify framework, such as FActScore, can be manipulated by adding obvious or repetitive subclaims to artificially inflate scores. |
| Approach: | They propose a decomposition-based tool called Core to filter subclaims based on their uniqueness and informativeness. |
| Outcome: | The proposed evaluation framework supports easy and modular use of Core and various decomposition strategies. |
FLASH: Focused Layer Attention Sink Hijacking (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) remain vulnerable to jailbreaking attacks despite advances in safety alignment . |
| Approach: | They propose a new diagnostic auditing framework that dismantles the model's internal safety anchor by precisely scaling attention scores in these vulnerable layers. |
| Outcome: | The proposed framework achieves a state-of-the-art Attack Success Rate of over 77% with an unprecedented efficiency of 1.53 queries on average. |
JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been used to mitigate misuse and to align with human values. |
| Approach: | They propose to use large-scale evaluations of various jailbreak attacks to identify key patterns and test them under eight advanced defenses. |
| Outcome: | The proposed attacks achieve high success rates but are easy to mitigate by defenses. |
ALRPHFS: Adversarially Learned Risk Patterns with Hierarchical Fast & Slow Reasoning for Robust Agent Defense (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing safety checks fail to capture complex semantic risks posed by harmful user inputs or unsafe agent behaviors. |
| Approach: | They propose a framework to bridge the semantic gap between safety checks and real-world risks. |
| Outcome: | The proposed framework achieves superior overall performance compared to existing baselines. |
PolitiCause: An Annotation Scheme and Corpus for Causality in Political Texts (2024.lrec-main)
Copied to clipboard
| Challenge: | PolitiCAUSE is a new corpus of political texts annotated for causality . it provides a detailed and robust annotation scheme for analyzing causal information . |
| Approach: | They propose a new corpus of political texts annotated for causality . they provide a detailed and robust annotation scheme for annotating causal information . |
| Outcome: | The proposed method achieves a moderate performance on the dataset, with a MCC score of 0.62. |
FCMR: Robust Evaluation of Financial Cross-Modal Multi-Hop Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for multimodal large language models lack data contamination and complex queries . financial cross-modal multi-hop reasoning is difficult to evaluate and requires precise cross-module reasoning . |
| Approach: | They propose a benchmark to analyze the reasoning capabilities of multimodal large language models. |
| Outcome: | The proposed model is categorized into three difficulty levels—easy, medium, and hard—for step-by-step evaluation. |
DPO Kernels: A Semantically-Aware, Kernel-Enhanced, and Divergence-Rich Paradigm for Direct Preference Optimization (2025.findings-acl)
Copied to clipboard
Amitava Das, Suranjana Trivedy, Danush Khanna, Yaswanth Narsupalli, Basab Ghosh, Rajarshi Roy, Gurpreet Singh, Vinija Jain, Vasu Sharma, Aishwarya Naresh Reganti, Aman Chadha
| Challenge: | Direct Preference Optimization (DPO) is a cornerstone for preference alignment but is constrained by fixed divergence measures and limited feature transformations. |
| Approach: | They propose a new enhancement of Direct Preference Optimization that integrates kernel methods to overcome these challenges. |
| Outcome: | The proposed model improves divergence measures and features by using kernels . the proposed model achieves state-of-the-art generalization in factuality, safety, reasoning, and instruction following . |
Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domains (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies that analyze unseen domains vary translation systems, annotators, or evaluation conditions, confounding domain effects with human annotation noise. |
| Approach: | They propose to use human error span annotations to evaluate translations of six translation systems across one seen news domain and two unseen technical domains to address these biases. |
| Outcome: | The proposed model improves on the human annotations in two unseen domains and on the news domains. |
Watermark under Fire: A Robustness Evaluation of LLM Watermarking (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Various watermarking methods have been proposed to identify LLM-generated texts . lack of unified evaluation platforms has left many critical questions unanswered . |
| Approach: | They systematize existing LLM watermarkers and watermark removal attacks and develop a unified platform that integrates them. |
| Outcome: | The proposed systematizes existing LLM watermarkers and watermark removal attacks, mapping out their design spaces. |
Debiasing Multilingual LLMs in Cross-lingual Latent Space (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages. |
| Approach: | They propose to perform debiasing in a joint latent space rather than directly on LLM representations by using an autoencoder trained on parallel TED talk scripts. |
| Outcome: | The proposed method improves both the overall debiasing performance and cross-lingual transferability of the proposed techniques across four languages. |
DualGuard: Dual-stream Large Language Model Watermarking Defense against Paraphrase and Spoofing Attack (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing watermarking algorithms focus on defending against paraphrase and piggyback spoofing attacks, which can inject harmful content, compromise reliability, and undermine trust in attribution. |
| Approach: | They propose an algorithm capable of defending against paraphrase and spoofing attacks. |
| Outcome: | Experiments on large language models and language models show that DualGuard is the first watermarking algorithm capable of defending against both paraphrase and spoofing attacks. |
CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for MLLMs are weak on explicit attacks, but weak on implicit ones. |
| Approach: | They propose an automated red-teaming pipeline that leverages reinforcement learning with tailored reward modules to generate diverse implicit samples across 14 domains. |
| Outcome: | The proposed method outperforms existing methods in implicit and explicit attacks while maintaining high utility. |
Disentangled Information Bottleneck for Adversarial Text Defense (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have proven that these deep models are super vulnerable to adversarial examples, which are slightly modified inputs. |
| Approach: | They propose a novel text defense method that separates the robust and non-robust features with a disentangled two-line framework rather than the one-line compression network in IB. |
| Outcome: | The proposed method outperforms six baselines on four datasets with accuracy improvements ranging from 3.8% to 20.7%. |
From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation (2026.findings-acl)
Copied to clipboard
Pujun Zheng, Jiacheng Yao, Jinquan Zheng, Chenyang Gu, Guoxiu He, Jiawei Liu, Yong Huang, Tianrui Guo, Wei Lu
| Challenge: | Large language models (LLMs) are currently used to evaluate scientific papers by assigning an absolute score to each paper independently. |
| Approach: | They propose a comparison-native framework for paper evaluation that integrates comparison into both data construction and model learning. |
| Outcome: | The proposed framework achieves an average relative improvement of 21.8% over the strong baseline DeepReview-14B, while exhibiting robust generalization to five previously unseen datasets. |
Misalignment Attack on Text-to-Image Models via Text Embedding Optimization and Inversion (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Text embedding is a key component of modern NLP models but also poses additional risks. |
| Approach: | They propose a framework that optimizes embeddings and inverts them to obtain misaligned prompts. |
| Outcome: | The proposed framework exploits the continuity and distribution characteristics of text embeddings to obtain misaligned prompts of discrete tokens. |
GRE Score: Generative Risk Evaluation for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models have revolutionized generative tasks, but concerns about their trustworthiness and vulnerability to adversarial attacks persist. |
| Approach: | They propose an attack-independent evaluation of LLM robustness using conditional generation for synthetic text creation and a method to quantify the model's resilience. |
| Outcome: | The proposed method achieves a consistent ranking of LLM robustness when compared to the attack-based model ranking on TrustLLM (CITATION). |
Exons-Detect: Identifying and Amplifying Exonic Tokens via Hidden-State Discrepancy for Robust AI-Generated Text Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for AI-generated text detection assume uniform token contributions, making them less robust under short sequences or localized token modifications. |
| Approach: | They propose a training-free method for AI-generated text detection based on an exon-aware token reweighting perspective. |
| Outcome: | The proposed method achieves state-of-the-art detection performance and robustness to adversarial attacks and varying input lengths. |
Topic-Based Watermarks for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing watermarking methods often involve trade-offs between attack robustness, generation quality and additional overhead. |
| Approach: | They propose a topic-guided watermarking scheme that partitions the vocabulary into topic-aligned token subsets. |
| Outcome: | The proposed method achieves text quality comparable to industry-leading systems and improves watermark robustness against paraphrasing and lexical perturbation attacks with minimal performance overhead. |
EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks (2026.findings-acl)
Copied to clipboard
Yijie Lu, Manman Zhao, Tianjie Ju, Zihe Yan, Xinbei Ma, Yuan Guo, Daizong Ding, Gongshen Liu, Zhuosheng Zhang
| Challenge: | Existing methods for red-teaming face a trade-off between requiring target-specific knowledge and incurring prohibitive computational costs. |
| Approach: | They propose a framework that evolves payloads exclusively on the semantic dimension via a discovery-deployment pipeline. |
| Outcome: | Experiments show that EVA outperforms baselines in terms of attack success rate while evolving benign seeds into successful attacks within 1.18 to 1.71 iterations. |
From Insight to Exploit: Leveraging LLM Collaboration for Adaptive Adversarial Text Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLMs can provide substantial zero-shot performance on diverse tasks, but it is crucial to assess their robustness against adversarial inputs. |
| Approach: | They introduce Static Deceptor and Dynamic Deceptr to generate adversarial examples . they produce subtle and natural-looking adversarials that preserve semantic similarity to text . |
| Outcome: | The proposed attacks are based on two LLM-based attacks that generate natural-looking examples that deceive the target LLM. |
CulturalBench: A Robust, Diverse and Challenging Benchmark for Measuring LMs’ Cultural Knowledge Through Human-AI Red-Teaming (2025.acl-long)
Copied to clipboard
Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, Chan Young Park, Shuyue Stella Li, Sahithya Ravi, Mehar Bhatia, Maria Antoniak, Yulia Tsvetkov, Vered Shwartz, Yejin Choi
| Challenge: | CulturalBench is a set of 1,696 human-written and human-verified questions to assess LMs’ cultural knowledge covering 45 global regions including underrepresented ones like Bangladesh, Zimbabwe, and Peru. |
| Approach: | They construct a set of 1,696 human-written and human-verified questions to assess LMs' cultural knowledge, covering 45 global regions including underrepresented ones like Bangladesh, Zimbabwe, and Peru. |
| Outcome: | The proposed model outperforms other models across cultures, while underperforming on questions related to North Africa, South America and Middle East. |
Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have not examined how backdoored agents can influence tool-use sequences to perform harmful actions. |
| Approach: | They propose a backdoor attack framework that embeds semantic triggers into fine-tuned LLM agents. |
| Outcome: | The proposed framework embeds semantic triggers into fine-tuned LLM agents . when triggered, the backdoored agent invokes memory-access tool calls to retrieve stored user context and exfiltrates it via disguised retrieval tool calls. |
Context-aware Watermark with Semantic Balanced Green-red Lists for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research suggests that watermarking methods cause degradation of text quality due to semantic disparities between the watermarked text and the unwatermarked text. |
| Approach: | They propose a semantic-aware watermark method that generates a watermark key considering contexts to split a green/red list for watermark injection. |
| Outcome: | The proposed method reduces performance drop due to adding bias on green lists . it also allows green lists to cover almost all semantics . |
MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming (2025.acl-long)
Copied to clipboard
| Challenge: | Existing jailbreak techniques rely on single-round interactions, pro-Corresponding author. |
| Approach: | They propose a multi-turn safety alignment framework to address the challenge of securing large language models in multi-round interactions. |
| Outcome: | The proposed framework exhibits state-of-the-art attack capabilities while improving safety performance on safety benchmarks. |
SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | SteerVLM is a lightweight steering module designed to guide Vision-Language Models (VLMs) towards outputs that better adhere to desired instructions. |
| Approach: | They propose a lightweight steering module that learns from latent embeddings of paired prompts encoding target and converse behaviors to dynamically adjust activations connecting the language modality with image context. |
| Outcome: | The proposed steering module outperforms existing intervention techniques on steering and hallucination mitigation benchmarks for VLMs. |
Don’t Say No: Jailbreaking LLM by Suppressing Refusal (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are vulnerable to "jailbreaking" attacks where crafted prompts manipulate them into producing toxic content. |
| Approach: | They propose to improve the target loss objective by combining a cosine decay schedule method with refusal suppression to achieve higher success rates. |
| Outcome: | The proposed approach outperforms baseline attacks and achieves state-of-the-art attack success rates. |
Forget the Unneeded: Backdooring Large Language Models via Contrastive-enhanced Machine Unlearning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning for Large Language Models find backdoor attacks to be significant in data-rich scenarios. |
| Approach: | They propose a backdoor attacks through contrastive-enhanced machine unlearning in data-limited scenarios . they use a machine un learning method to capture precise backdoor patterns . |
| Outcome: | The proposed method captures precise backdoor patterns without association between triggers and backdoors, reducing side effects. |
VLA-Mark: A cross modal watermark for large vision-language alignment models (2025.emnlp-main)
Copied to clipboard
Shuliang Liu, Zheng Qi, Jesse Jiaxi Xu, Yibo Yan, Junyan Zhang, He Geng, Aiwei Liu, Peijie Jiang, Jia Liu, Yik-Cheung Tam, Xuming Hu
| Challenge: | Existing text watermarking methods disrupt visual-textual alignment, leaving semantic-critical concepts vulnerable. |
| Approach: | They propose a vision-aligned framework that embeds detectable watermarks into outputs . they combine localized patch affinity, global semantic coherence, contextual attention patterns . |
| Outcome: | The proposed framework shows lower PPL and higher BLEU than conventional methods with near-perfect detection (98.8% AUC). |
SynTOD: Augmented Response Synthesis for Robust End-to-End Task-Oriented Dialogue System (2024.lrec-main)
Copied to clipboard
| Challenge: | Task-oriented dialogue systems focus on training multiple tasks such as language understanding, tracking states, and generating appropriate responses to help users achieve their specific goals. |
| Approach: | They exploit the ability of pre-trained models to provide synthesis responses for fine-tuning end-to-end TOD systems. |
| Outcome: | The proposed model outperforms baseline models on multiwoz datasets and is available for further exploitation. |
MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation (2026.acl-long)
Copied to clipboard
Dawei Wang, Di Zhao, Xinyuan Liu, Marci Chi Ma, Xiaoyang Liu, Chengming Zhou, Gary Ushaw, Richard Davison
| Challenge: | Embodied AI systems are open, where agents may leave or enter mid-task due to hardware failures or task-related errors. |
| Approach: | They propose a framework that reformulates credit assignment as a rank aggregation problem using contribution-based pairwise comparisons among agents generated by large multimodal models. |
| Outcome: | The proposed framework can guide agents toward effective cooperation in complex tasks of different types. |
The ParlaSent Multilingual Training Dataset for Sentiment Identification in Parliamentary Proceedings (2024.lrec-main)
Copied to clipboard
| Challenge: | The paper presents a new training dataset of sentences in 7 languages, manually annotated for sentiment, which is used in a series of experiments focused on training a robust sentiment identifier for parliamentary proceedings. |
| Approach: | They propose to use a dataset of sentences manually annotated for sentiment to train a robust sentiment identifier for parliamentary proceedings. |
| Outcome: | The proposed model performs very well on languages not seen during fine-tuning and additional fine- tuning data from other languages significantly improves the target parliament’s results. |
Beyond Explicit Refusals: Soft-Failure Attacks on Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing jamming attacks on RAG systems typically induce explicit refusals or denial-of-service behaviors. |
| Approach: | They propose a black-box attack framework that exploits safety-aligned behaviors of large language models to trigger soft failures. |
| Outcome: | The proposed framework exploits safety-aligned behaviors of large language models to induce soft failures. |
Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used in education, yet their default usefulness conflicts with pedagogical principles. |
| Approach: | They propose an adversarial student agent that they fine-tune to jailbreak LLM-based tutors and propose a benchmark to evaluate tutor robustness. |
| Outcome: | The proposed model fine-tunes to jailbreak LLM-based tutors, and shows that they perform well under adversarial student attacks. |
QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models that use multimodal inputs are often noisy or incomplete. |
| Approach: | They propose a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via aleatoric uncertainty. |
| Outcome: | The proposed framework is competitive or state-of-the-art across diverse degradation scenarios and exhibits a promising One-Checkpoint-for-all property in practice. |
Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LLMs) have demonstrated impressive performances across diverse domains, but how their safety benefits from enhanced reasoning capabilities against jailbreak queries remains unexplored. |
| Approach: | They propose a safety-aware reasoning paradigm that integrates a pivot token-based safety-based reasoning mechanism into LLMs’ generation process. |
| Outcome: | The proposed model improves the safety of large language models against jailbreak queries while minimizing attacks and maintaining the original performance. |
HORIZON: A Benchmark for In-the-wild User Behaviour Modeling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing user modeling benchmarks focus on short sessions and next-item prediction within a single domain. |
| Approach: | They propose a benchmark that reformulates user modeling along three axes . it covers 54M users and 35M items, enabling pretraining and evaluation . they propose tasks and evaluation setups that better reflect real-world deployment scenarios . |
| Outcome: | The proposed benchmark covers 54M users and 35M items, and is based on Amazon Reviews. |
Learning on Imbalanced Noisy Data via Debiased Sample Selection and LLM-Driven Annotation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to learning with noisy labels are prone to selection bias and training bias . obtaining large-scale high-quality datasets is expensive and time-consuming in practical scenarios . |
| Approach: | They propose an imbalanced learning with noisy labels task to let model learn from noisy labels . they first conduct debiased sample selection to better separate clean samples from noisy samples . then they feed selected clean samples to active annotator large language models for re-annotating noisy samples. |
| Outcome: | The proposed method is superior to existing methods on synthetic and real-world datasets. |
NeuralFSM: Adaptive Multi-Agent Coordination via Learning Finite-State Execution Policy (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multi-agent problem solving rely on hand-crafted protocols or automatically designed topologies. |
| Approach: | They propose a state-driven framework that formulates multi-agent problem solving as a finite-state execution process. |
| Outcome: | The proposed framework outperforms baselines on diverse benchmarks by 6.74%–19.39% while reducing token consumption. |
Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event extraction are limited in their ability to recall nuanced or rare events. |
| Approach: | They propose a hybrid approach that leverages a self-mixture of agents and a discriminative sequence tagger to resolve ambiguities and enhance overall event prediction quality. |
| Outcome: | The proposed approach outperforms existing state-of-the-art methods across three benchmark datasets. |
MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Knowledge Poisoning Attacks (2026.acl-long)
Copied to clipboard
Hyeonjeong Ha, Qiusi Zhan, Jeonghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng, Kai-Wei Chang, Daniel Kang, Heng Ji
| Challenge: | Existing research exposes multimodal large language models to knowledge poisoning attacks . localized poisoning attack achieves up to 56% success rate even under restricted access . globalized poison attack completely disrupts model generation to 0% accuracy with just one poisoned content. |
| Approach: | They propose a framework to study the vulnerability of multimodal RAG under knowledge poisoning attacks. |
| Outcome: | The proposed framework exploits two new attack strategies on multimodal RAGs under knowledge poisoning. |
More Thinking, Less Talking: Internalizing Deliberative Safety into LLM Parameters (2026.acl-long)
Copied to clipboard
| Challenge: | Existing safety alignment methods leave Large Language Models vulnerable to sophisticated jailbreak attacks. |
| Approach: | They propose a safety reasoning internalization framework that internalizes safety reasoning into an implicit computational pathway using Low-Rank Adaptation (LoRA). |
| Outcome: | The proposed framework achieves a 43% lower Attack Success Rate (ASR) against distinct jailbreak attacks compared to strong baselines. |
ConSensus: Multi-Agent Collaboration for Multimodal Sensing (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are increasingly grounded in sensor data to perceive and reason about human physiology and the physical world. |
| Approach: | They propose a training-free multi-agent collaboration framework that decomposes multimodal sensing tasks into specialized, modality-aware agents. |
| Outcome: | The proposed framework matches or exceeds debate methods on multimodal sensing benchmarks while achieving 12.7 times reduction in token cost. |
Chimera: Compositional Jailbreak Attacks on LLMs via Judgment-Driven Search over Heterogeneous Strategies (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for evaluating large language models face two limitations: they explore homogeneous transformations in isolation and rely on brittle judgment metrics that misclassify non-refusal hallucinations as successful attacks. |
| Approach: | They propose a framework that generates compositional jailbreak attacks via judgment-driven search over heterogeneous strategies. |
| Outcome: | The proposed framework generates compositional jailbreak attacks over heterogeneous strategies . strongREJECT++ improves attack success rates and transferability compared to state-of-the-art . |
CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward (2025.emnlp-main)
Copied to clipboard
Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F. Wong, Songyang Zhang, Kai Chen
| Challenge: | Existing approaches lack robustness to handle complex edge cases and generalizability across different domains. |
| Approach: | They develop an accurate and lightweight verifier model for evaluation and outcome reward that matches unstructured outputs against standard answers. |
| Outcome: | The proposed model can process multiple answer types including multi-subproblems, formulas, and sequence answers while identifying abnormal/invalid responses. |
Better LLM Reasoning via Dual-Play (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made remarkable progress through Reinforcement Learning with Verifiable Rewards (RLVR) however, external supervision remains a bottleneck for tasks and domains for which supervised data are scarce or non-existent. |
| Approach: | They propose a novel dual-play framework that adversarially trains two models initialized from the same base model. |
| Outcome: | The proposed framework improves the math reasoning performance of large language models. |
Fico: Evaluating Vision-Language Models under Visual Fidelity and Compression at Scale (2026.findings-acl)
Copied to clipboard
| Challenge: | Visual text compression is emerging paradigm for rendering text as images for processing by vision-language models. |
| Approach: | They propose a benchmark to assess VLM robustness under dense visual inputs. |
| Outcome: | Evaluating 13 general-purpose VLMs and 3 OCR-specialized models reveals performance drops sharply under increased density or reduced resolution; cross-task transfer between OCR, NIAH, and VQA is limited; and VQ is comparatively robust because low-level details are lost before high-level semantics. |
DetectRL-X: Towards Reliable Multilingual and Real-World LLM-Generated Text Detection (2026.acl-long)
Copied to clipboard
Junchao Wu, Yefeng Liu, Chenyu Zhu, Hao Zhang, Zeyu Wu, Tianqi Shi, Yichao Du, Longyue Wang, Weihua Luo, Jinsong Su, Derek F. Wong
| Challenge: | Existing detectors are limited in their ability to detect large language models generated content in multilingual environments. |
| Approach: | They propose a multilingual benchmark to evaluate advanced detectors across 8 dimensions to better align with real-world applications. |
| Outcome: | The proposed benchmark encompasses 8 languages commonly used in commercial contexts and collects human-written texts from 6 domains highly susceptible to LLM misuse. |
Retracing the Past: LLMs Emit Training Data When They Get Lost (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extracting training data from large language models exhibit limited success . existing methods offer limited insight into the fundamental drivers of memorization leakage . |
| Approach: | They propose a framework for extracting memorized data by maximizing model uncertainty . they propose mismatched fine-tuning to weaken alignment and induce confusion . |
| Outcome: | The proposed attacks outperform baselines on unaligned and aligned LLMs . the proposed attacks exploit the model uncertainty of the input snippets induced by the model entropy spike . |
Beyond Static Synthetic Noise: Assessing the Robustness of Large Language Models to Natural Context Variation in the Real World (2026.findings-acl)
Copied to clipboard
| Challenge: | Current robustness evaluation methods rely on static synthetic perturbations to stress-test models. |
| Approach: | They propose a framework for automatically evaluating QA models under naturally occurring textual perturbations by replacing context passages with revised Wikipedia edit histories. |
| Outcome: | The proposed framework replaces context passages with revised Wikipedia edit histories to improve model performance. |
BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing supervised defense methods rely on labeled malicious agents to train a supervised model of malicious behavior. |
| Approach: | They propose an unsupervised defense method that learns without requiring any attack-specific labels or prior knowledge of malicious behaviors. |
| Outcome: | The proposed method detects diverse attack types across MAS with various communication patterns while maintaining superior generalizability compared to baselines. |
B-APO: Bias-Targeted Adversarial Preference Optimization for Debiasing Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing debiasing methods create biased responses by completely removing an entire modality, forming an extreme and static training environment. |
| Approach: | They propose a method to debiase multimodal large language models by masking one modality and then enlarge the margin between clean and adversarial responses. |
| Outcome: | The proposed method achieves superior debiasing performance while maintaining general capabilities. |
How Do LLMs "Trust" Unknown Knowledge? An Unknown Knowledge Based Jailbreak Attack (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on how to effectively utilize unknown knowledge has focused on how it can be used to enhance LLMs' performance in specialized fields. |
| Approach: | They propose a completely unrestricted and fully randomized jailbreak attack that embeds malicious queries within trust-enhanced unknown knowledge. |
| Outcome: | The proposed method achieves 99% to 100% ASR on all tested LLMs, including the latest GPT-5.1, and becomes SOTA. |
Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors (2026.acl-long)
Copied to clipboard
Zhiwei Zhang, Fei Zhao, Rui Wang, Zezhong Wang, Bin Liang, Jiakang Wang, Yao Hu, Shaosheng Cao, Kam-Fai Wong
| Challenge: | Large language models (LLMs) can call tools effectively, but they remain brittle in multi-turn execution. |
| Approach: | They propose a framework that converts execution errors into on-policy corrective supervision within the RL training loop. |
| Outcome: | The proposed framework improves the error recovery rate of Qwen3-8B by 5.7% absolute and overall accuracy by 4.0% on BFCL v4 Multi-Turn. |
Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Multi-agent LLMs are rapidly moving from prototype to real-world use . network topology is a first-order security parameter in multi-aggent systems . |
| Approach: | They propose a framework for comparing topology-conditioned memory leakage in multi-agent LLM systems. |
| Outcome: | The proposed framework evaluates topology-conditioned memory leakage in multi-agent LLM systems. |
ReTRE: Benchmarking LLM Transfer Robustness with Structure-Preserving Variants (2026.acl-long)
Copied to clipboard
| Challenge: | Learning transfer theory emphasizes that applying acquired knowledge to novel manifestations is a key signal of deep understanding |
| Approach: | They propose a benchmark that probes transfer robustness along two rewrite levels: Near Transfer and Far Transfer. |
| Outcome: | The proposed benchmark demonstrates that large language models are robust when faced with novel manifestations of the same problem. |
ToM-Synth: Scaling Robust Theory of Mind in LLMs via 6,912 Structured Social Units (2026.findings-acl)
Copied to clipboard
Guiyang Hou, Xiang Huang, Shangke Lyu, Yuchuan Wu, Weiyao Luo, Xinyu Mei, Yongliang Shen, Weiming Lu, Yongbin Li
| Challenge: | Existing methods endowing LLMs with Theory of Mind fail to internalize the augmented ToM into the LLM. |
| Approach: | They propose a factorial combinatorial synthesis framework that enables systematic synthesis of ToM data and uses it for RL fine-tuning. |
| Outcome: | The proposed framework yields a training dataset of 27,648 instances. |
Dictionary Guided Sparse Logit Editing for Reliable Jailbreak Attacks (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to optimize large language models suffer from high computational costs and produce uninterpretable, high-perplexity inputs. |
| Approach: | They propose a sparse index-based intervention that bypasses guardrails via sparser logit editing. |
| Outcome: | The proposed method bypasses guardrails by modifying pre-softmax logits without gradients or auxiliary models. |